2024-09-10T20:48:57.241 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T20:48:57.264 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T20:48:57.372 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899652 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/cap-flush} email: hyelloji@redhat.com first_in_suite: false job_id: '7899652' kernel: client: sha1: distro flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - Replacing daemon mds.a sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 kclient: syntax: v2 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi098.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBERnTf0dpCtnGZSeXc+jviUtasaLkYOu1WhRA8LTXF1C52RVaVi8QT9PLudcJarM74AlqUqrrTmkxgS+h8G6/lg= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFQiF0McfLgNBxTrH+/J5mvaNf8W/eSVcmYR9y3gnU3TbDAatBiMJUglgn7cPBh9lCfJ378lRhED+Qbz5Fv2Mc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_cap_flush teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T20:48:57.372 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T20:48:57.373 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T20:48:57.373 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T20:48:57.375 INFO:teuthology.task.internal:Checking packages... 2024-09-10T20:48:57.396 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T20:48:57.396 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T20:48:57.396 INFO:teuthology.packaging:ref: None 2024-09-10T20:48:57.396 INFO:teuthology.packaging:tag: None 2024-09-10T20:48:57.396 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T20:48:57.396 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:48:57.396 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T20:48:57.546 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854.g8f219c47 2024-09-10T20:48:57.548 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T20:48:57.560 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T20:48:57.560 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T20:48:57.593 INFO:teuthology.task.internal:Saving configuration 2024-09-10T20:48:57.611 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T20:48:57.628 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T20:48:57.661 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 20:43:52.850565', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBERnTf0dpCtnGZSeXc+jviUtasaLkYOu1WhRA8LTXF1C52RVaVi8QT9PLudcJarM74AlqUqrrTmkxgS+h8G6/lg='} 2024-09-10T20:48:57.692 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 20:43:52.851929', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFQiF0McfLgNBxTrH+/J5mvaNf8W/eSVcmYR9y3gnU3TbDAatBiMJUglgn7cPBh9lCfJ378lRhED+Qbz5Fv2Mc='} 2024-09-10T20:48:57.692 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T20:48:57.702 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-10T20:48:57.703 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-10T20:48:57.703 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T20:48:57.801 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbb6445fbe0>, signals=[15]) 2024-09-10T20:48:57.801 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T20:48:57.813 INFO:teuthology.task.internal:Opening connections... 2024-09-10T20:48:57.813 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T20:48:57.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:48:57.899 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-09-10T20:48:57.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:48:57.983 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T20:48:57.996 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2024-09-10T20:48:58.014 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2024-09-10T20:48:58.014 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2024-09-10T20:48:58.073 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2024-09-10T20:48:58.073 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="9" 2024-09-10T20:48:58.073 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2024-09-10T20:48:58.073 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2024-09-10T20:48:58.074 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="9" 2024-09-10T20:48:58.074 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el9" 2024-09-10T20:48:58.074 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T20:48:58.074 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2024-09-10T20:48:58.074 INFO:teuthology.orchestra.run.smithi098.stdout:LOGO="fedora-logo-icon" 2024-09-10T20:48:58.074 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T20:48:58.074 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2024-09-10T20:48:58.074 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T20:48:58.074 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T20:48:58.074 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T20:48:58.075 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2024-09-10T20:48:58.101 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-09-10T20:48:58.119 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-09-10T20:48:58.119 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-09-10T20:48:58.177 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2024-09-10T20:48:58.177 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9" 2024-09-10T20:48:58.177 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2024-09-10T20:48:58.177 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2024-09-10T20:48:58.177 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9" 2024-09-10T20:48:58.178 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9" 2024-09-10T20:48:58.178 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T20:48:58.178 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2024-09-10T20:48:58.178 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon" 2024-09-10T20:48:58.178 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T20:48:58.178 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2024-09-10T20:48:58.178 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T20:48:58.178 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T20:48:58.178 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T20:48:58.178 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-09-10T20:48:58.209 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T20:48:58.224 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T20:48:58.252 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T20:48:58.253 DEBUG:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2024-09-10T20:48:58.255 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-09-10T20:48:58.272 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T20:48:58.287 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T20:48:58.287 DEBUG:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2024-09-10T20:48:58.312 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-09-10T20:48:58.378 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T20:48:58.452 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T20:48:58.471 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T20:48:58.471 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-09-10T20:48:58.472 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-09-10T20:48:58.472 DEBUG:teuthology.orchestra.run.smithi098:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T20:48:58.472 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T20:48:58.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:48:58.490 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-09-10T20:48:58.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:48:58.491 DEBUG:teuthology.orchestra.run.smithi098:> uname -r 2024-09-10T20:48:58.549 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-503.el9.x86_64 2024-09-10T20:48:58.549 INFO:teuthology.task.kernel:Running kernel on smithi190: 5.14.0-503.el9.x86_64 2024-09-10T20:48:58.549 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum install -y kernel 2024-09-10T20:48:58.550 INFO:teuthology.orchestra.run.smithi098.stdout:5.14.0-503.el9.x86_64 2024-09-10T20:48:58.550 INFO:teuthology.task.kernel:Running kernel on smithi098: 5.14.0-503.el9.x86_64 2024-09-10T20:48:58.550 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum install -y kernel 2024-09-10T20:49:01.178 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-10T20:49:02.912 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-10T20:49:04.418 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.2 MB 00:04 2024-09-10T20:49:06.453 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-10T20:49:08.246 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-10T20:49:09.930 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-10T20:49:11.045 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 6.5 MB 00:00 2024-09-10T20:49:11.401 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-10T20:49:12.946 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 19 kB 00:00 2024-09-10T20:49:14.250 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-10T20:49:16.801 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-10T20:49:18.846 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T20:49:18.846 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T20:49:18.926 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-09-10T20:49:18.938 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-10T20:49:18.938 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2024-09-10T20:49:18.938 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-10T20:49:18.938 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-09-10T20:49:18.938 INFO:teuthology.orchestra.run.smithi190.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T20:49:18.938 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-09-10T20:49:18.938 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T20:49:18.939 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T20:49:18.939 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T20:49:18.939 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:49:18.939 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-09-10T20:49:18.939 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-09-10T20:49:18.939 INFO:teuthology.orchestra.run.smithi190.stdout:Install 4 Packages 2024-09-10T20:49:18.939 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:49:18.939 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 79 M 2024-09-10T20:49:18.939 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 126 M 2024-09-10T20:49:18.939 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-09-10T20:49:19.366 INFO:teuthology.orchestra.run.smithi190.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 176 kB/s | 23 kB 00:00 2024-09-10T20:49:19.656 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-10T20:49:20.816 INFO:teuthology.orchestra.run.smithi190.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 9.9 MB/s | 16 MB 00:01 2024-09-10T20:49:21.474 INFO:teuthology.orchestra.run.smithi190.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 14 MB/s | 29 MB 00:02 2024-09-10T20:49:21.663 INFO:teuthology.orchestra.run.smithi098.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T20:49:21.663 INFO:teuthology.orchestra.run.smithi098.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T20:49:21.744 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-10T20:49:21.756 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-10T20:49:21.756 INFO:teuthology.orchestra.run.smithi098.stdout: Package Architecture Version Repository Size 2024-09-10T20:49:21.756 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-10T20:49:21.756 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-10T20:49:21.756 INFO:teuthology.orchestra.run.smithi098.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T20:49:21.756 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-10T20:49:21.756 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T20:49:21.756 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T20:49:21.756 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T20:49:21.756 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:49:21.756 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-10T20:49:21.757 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-09-10T20:49:21.757 INFO:teuthology.orchestra.run.smithi098.stdout:Install 4 Packages 2024-09-10T20:49:21.757 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:49:21.757 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 79 M 2024-09-10T20:49:21.757 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 126 M 2024-09-10T20:49:21.757 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-10T20:49:21.867 INFO:teuthology.orchestra.run.smithi190.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 13 MB/s | 35 MB 00:02 2024-09-10T20:49:21.867 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:49:21.867 INFO:teuthology.orchestra.run.smithi190.stdout:Total 27 MB/s | 79 MB 00:02 2024-09-10T20:49:21.991 INFO:teuthology.orchestra.run.smithi098.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 194 kB/s | 23 kB 00:00 2024-09-10T20:49:22.553 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-09-10T20:49:22.943 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-09-10T20:49:22.943 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-09-10T20:49:23.508 INFO:teuthology.orchestra.run.smithi098.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 19 MB/s | 29 MB 00:01 2024-09-10T20:49:23.902 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-09-10T20:49:23.902 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-09-10T20:49:24.233 INFO:teuthology.orchestra.run.smithi098.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 15 MB/s | 35 MB 00:02 2024-09-10T20:49:24.559 INFO:teuthology.orchestra.run.smithi098.stdout:(4/4): kernel-core-5.14.0-505.el9.x86_64.rpm 5.8 MB/s | 16 MB 00:02 2024-09-10T20:49:24.559 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:49:24.559 INFO:teuthology.orchestra.run.smithi098.stdout:Total 28 MB/s | 79 MB 00:02 2024-09-10T20:49:25.236 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-10T20:49:25.595 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-10T20:49:25.595 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-10T20:49:26.374 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-09-10T20:49:26.588 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-10T20:49:26.588 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-10T20:49:27.054 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T20:49:27.150 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T20:49:29.046 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-10T20:49:29.683 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T20:49:29.777 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T20:49:29.939 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T20:49:30.078 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T20:49:32.563 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T20:49:32.722 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T20:49:34.912 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T20:49:34.953 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:49:37.525 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T20:49:37.565 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:49:39.892 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:49:42.384 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:50:18.378 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:50:18.417 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:50:18.746 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:50:18.747 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T20:50:18.747 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T20:50:18.747 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T20:50:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:50:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:50:19.560 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-09-10T20:50:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T20:50:19.560 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T20:50:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T20:50:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T20:50:19.561 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:50:19.561 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-09-10T20:50:19.918 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T20:50:20.007 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T20:50:20.007 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T20:50:20.007 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T20:50:20.007 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T20:50:20.007 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T20:50:20.007 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-09-10T20:50:20.007 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi190.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T20:50:20.008 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T20:50:20.065 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T20:50:20.065 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T20:50:20.065 DEBUG:teuthology.orchestra.run.smithi190:> sudo rpm -qi grub2-tools 2024-09-10T20:50:20.101 INFO:teuthology.orchestra.run.smithi190.stdout:Name : grub2-tools 2024-09-10T20:50:20.101 INFO:teuthology.orchestra.run.smithi190.stdout:Epoch : 1 2024-09-10T20:50:20.101 INFO:teuthology.orchestra.run.smithi190.stdout:Version : 2.06 2024-09-10T20:50:20.101 INFO:teuthology.orchestra.run.smithi190.stdout:Release : 82.el9 2024-09-10T20:50:20.101 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: x86_64 2024-09-10T20:50:20.101 INFO:teuthology.orchestra.run.smithi190.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T20:50:20.101 INFO:teuthology.orchestra.run.smithi190.stdout:Group : Unspecified 2024-09-10T20:50:20.101 INFO:teuthology.orchestra.run.smithi190.stdout:Size : 8274670 2024-09-10T20:50:20.102 INFO:teuthology.orchestra.run.smithi190.stdout:License : GPLv3+ 2024-09-10T20:50:20.102 INFO:teuthology.orchestra.run.smithi190.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T20:50:20.102 INFO:teuthology.orchestra.run.smithi190.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T20:50:20.102 INFO:teuthology.orchestra.run.smithi190.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T20:50:20.102 INFO:teuthology.orchestra.run.smithi190.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T20:50:20.102 INFO:teuthology.orchestra.run.smithi190.stdout:Packager : builder@centos.org 2024-09-10T20:50:20.102 INFO:teuthology.orchestra.run.smithi190.stdout:Vendor : CentOS 2024-09-10T20:50:20.102 INFO:teuthology.orchestra.run.smithi190.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T20:50:20.102 INFO:teuthology.orchestra.run.smithi190.stdout:Summary : Support tools for GRUB. 2024-09-10T20:50:20.103 INFO:teuthology.orchestra.run.smithi190.stdout:Description : 2024-09-10T20:50:20.103 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:50:20.103 INFO:teuthology.orchestra.run.smithi190.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T20:50:20.103 INFO:teuthology.orchestra.run.smithi190.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T20:50:20.103 INFO:teuthology.orchestra.run.smithi190.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T20:50:20.103 INFO:teuthology.orchestra.run.smithi190.stdout:hardware devices. 2024-09-10T20:50:20.103 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:50:20.103 INFO:teuthology.orchestra.run.smithi190.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T20:50:20.104 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T20:50:20.104 INFO:teuthology.task.kernel:Updating grub on smithi190 to boot 5.14.0-505.el9.x86_64 2024-09-10T20:50:20.104 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T20:50:20.703 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2024-09-10T20:50:20.897 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:50:20.936 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:50:21.300 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:50:21.300 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T20:50:21.301 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T20:50:21.301 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T20:50:21.986 INFO:teuthology.orchestra.run.smithi190.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T20:50:22.014 INFO:teuthology.orchestra.run.smithi190.stderr:done 2024-09-10T20:50:22.016 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-09-10T20:50:22.032 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.3wjEj7vLGx 2024-09-10T20:50:22.033 DEBUG:teuthology.orchestra.run.smithi190:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3wjEj7vLGx 2024-09-10T20:50:22.100 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.3wjEj7vLGx 2024-09-10T20:50:22.104 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:50:22.104 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:50:22.104 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-10T20:50:22.104 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T20:50:22.104 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T20:50:22.104 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T20:50:22.104 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T20:50:22.104 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:50:22.104 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-10T20:50:22.237 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.3wjEj7vLGx is 6KB 2024-09-10T20:50:22.258 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.3wjEj7vLGx 2024-09-10T20:50:22.273 DEBUG:teuthology.orchestra.run.smithi190:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T20:50:22.336 INFO:teuthology.orchestra.run.smithi190.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T20:50:22.336 INFO:teuthology.orchestra.run.smithi190.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T20:50:22.336 INFO:teuthology.orchestra.run.smithi190.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T20:50:22.336 INFO:teuthology.orchestra.run.smithi190.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T20:50:22.339 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T20:50:22.469 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T20:50:22.476 DEBUG:teuthology.orchestra.run.smithi190:> sudo shutdown -r now 2024-09-10T20:50:22.541 INFO:teuthology.orchestra.run.smithi098.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T20:50:22.541 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T20:50:22.541 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T20:50:22.541 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T20:50:22.541 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T20:50:22.541 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-10T20:50:22.541 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi098.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T20:50:22.542 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T20:50:22.600 INFO:teuthology.orchestra.run.smithi098.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T20:50:22.600 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T20:50:22.600 DEBUG:teuthology.orchestra.run.smithi098:> sudo rpm -qi grub2-tools 2024-09-10T20:50:22.642 INFO:teuthology.orchestra.run.smithi098.stdout:Name : grub2-tools 2024-09-10T20:50:22.642 INFO:teuthology.orchestra.run.smithi098.stdout:Epoch : 1 2024-09-10T20:50:22.642 INFO:teuthology.orchestra.run.smithi098.stdout:Version : 2.06 2024-09-10T20:50:22.642 INFO:teuthology.orchestra.run.smithi098.stdout:Release : 82.el9 2024-09-10T20:50:22.642 INFO:teuthology.orchestra.run.smithi098.stdout:Architecture: x86_64 2024-09-10T20:50:22.642 INFO:teuthology.orchestra.run.smithi098.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T20:50:22.642 INFO:teuthology.orchestra.run.smithi098.stdout:Group : Unspecified 2024-09-10T20:50:22.643 INFO:teuthology.orchestra.run.smithi098.stdout:Size : 8274670 2024-09-10T20:50:22.643 INFO:teuthology.orchestra.run.smithi098.stdout:License : GPLv3+ 2024-09-10T20:50:22.643 INFO:teuthology.orchestra.run.smithi098.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T20:50:22.643 INFO:teuthology.orchestra.run.smithi098.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T20:50:22.643 INFO:teuthology.orchestra.run.smithi098.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T20:50:22.643 INFO:teuthology.orchestra.run.smithi098.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T20:50:22.643 INFO:teuthology.orchestra.run.smithi098.stdout:Packager : builder@centos.org 2024-09-10T20:50:22.643 INFO:teuthology.orchestra.run.smithi098.stdout:Vendor : CentOS 2024-09-10T20:50:22.643 INFO:teuthology.orchestra.run.smithi098.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T20:50:22.643 INFO:teuthology.orchestra.run.smithi098.stdout:Summary : Support tools for GRUB. 2024-09-10T20:50:22.643 INFO:teuthology.orchestra.run.smithi098.stdout:Description : 2024-09-10T20:50:22.643 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:50:22.643 INFO:teuthology.orchestra.run.smithi098.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T20:50:22.644 INFO:teuthology.orchestra.run.smithi098.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T20:50:22.644 INFO:teuthology.orchestra.run.smithi098.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T20:50:22.644 INFO:teuthology.orchestra.run.smithi098.stdout:hardware devices. 2024-09-10T20:50:22.644 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:50:22.644 INFO:teuthology.orchestra.run.smithi098.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T20:50:22.645 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T20:50:22.645 INFO:teuthology.task.kernel:Updating grub on smithi098 to boot 5.14.0-505.el9.x86_64 2024-09-10T20:50:22.645 DEBUG:teuthology.orchestra.run.smithi098:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T20:50:23.330 INFO:teuthology.orchestra.run.smithi098.stderr:Generating grub configuration file ... 2024-09-10T20:50:24.621 INFO:teuthology.orchestra.run.smithi098.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T20:50:24.649 INFO:teuthology.orchestra.run.smithi098.stderr:done 2024-09-10T20:50:24.651 DEBUG:teuthology.orchestra.run.smithi098:> mktemp 2024-09-10T20:50:24.666 INFO:teuthology.orchestra.run.smithi098.stdout:/tmp/tmp.luSNzDOnhT 2024-09-10T20:50:24.667 DEBUG:teuthology.orchestra.run.smithi098:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.luSNzDOnhT 2024-09-10T20:50:24.734 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0666 /tmp/tmp.luSNzDOnhT 2024-09-10T20:50:24.878 DEBUG:teuthology.orchestra.remote:smithi098:/tmp/tmp.luSNzDOnhT is 6KB 2024-09-10T20:50:24.899 DEBUG:teuthology.orchestra.run.smithi098:> rm -fr /tmp/tmp.luSNzDOnhT 2024-09-10T20:50:24.914 DEBUG:teuthology.orchestra.run.smithi098:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T20:50:24.976 INFO:teuthology.orchestra.run.smithi098.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T20:50:24.977 INFO:teuthology.orchestra.run.smithi098.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T20:50:24.977 INFO:teuthology.orchestra.run.smithi098.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T20:50:24.977 INFO:teuthology.orchestra.run.smithi098.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T20:50:24.978 DEBUG:teuthology.orchestra.run.smithi098:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T20:50:25.136 DEBUG:teuthology.orchestra.run.smithi098:> sudo shutdown -r now 2024-09-10T20:50:52.505 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-10T20:50:52.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-10T20:50:52.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:50:55.144 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T20:50:55.144 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-10T20:50:55.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:51:11.037 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-09-10T20:51:13.597 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-09-10T20:51:20.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-10T20:51:20.045 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:51:22.600 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-10T20:51:22.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:51:23.133 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-09-10T20:51:25.661 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-09-10T20:51:35.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-10T20:51:35.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:51:37.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-10T20:51:37.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:51:53.533 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-09-10T20:51:56.093 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-09-10T20:52:08.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-10T20:52:08.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:52:08.857 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-09-10T20:52:09.146 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-10T20:52:09.146 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-505.el9.x86_64"... 2024-09-10T20:52:09.147 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-09-10T20:52:09.162 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-505.el9.x86_64 2024-09-10T20:52:09.162 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T20:52:09.162 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T20:52:09.162 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T20:52:10.163 DEBUG:teuthology.parallel:result is None 2024-09-10T20:52:11.095 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-10T20:52:11.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:52:11.438 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-09-10T20:52:11.783 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-10T20:52:11.783 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-10T20:52:11.784 DEBUG:teuthology.orchestra.run.smithi098:> uname -r 2024-09-10T20:52:11.799 INFO:teuthology.orchestra.run.smithi098.stdout:5.14.0-505.el9.x86_64 2024-09-10T20:52:11.799 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T20:52:11.799 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T20:52:11.799 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T20:52:12.801 DEBUG:teuthology.parallel:result is None 2024-09-10T20:52:12.801 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T20:52:12.807 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T20:52:12.808 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T20:52:12.810 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T20:52:12.854 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T20:52:12.860 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T20:52:12.876 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T20:52:12.876 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T20:52:12.880 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T20:52:12.939 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T20:52:12.945 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T20:52:12.946 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T20:52:12.948 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T20:52:13.044 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:52:13.066 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:52:13.073 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:52:13.166 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:52:13.168 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T20:52:13.176 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T20:52:13.176 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T20:52:13.179 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T20:52:13.223 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T20:52:13.232 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T20:52:13.233 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T20:52:13.235 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T20:52:13.276 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T20:52:13.354 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T20:52:13.425 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T20:52:13.520 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T20:52:13.578 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:52:13.579 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T20:52:13.638 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T20:52:13.693 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T20:52:13.739 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T20:52:13.829 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T20:52:13.866 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:52:13.866 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T20:52:13.934 DEBUG:teuthology.orchestra.run.smithi098:> sudo service rsyslog restart 2024-09-10T20:52:13.937 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-09-10T20:52:13.980 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T20:52:14.028 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T20:52:14.534 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T20:52:14.543 INFO:teuthology.task.internal:Starting timer... 2024-09-10T20:52:14.543 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T20:52:14.579 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T20:52:14.603 DEBUG:teuthology.orchestra.run.smithi098:> sudo service auditd rotate 2024-09-10T20:52:14.656 INFO:teuthology.orchestra.run.smithi098.stdout:Rotating logs: 2024-09-10T20:52:14.658 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2024-09-10T20:52:14.723 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2024-09-10T20:52:14.725 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T20:52:14.726 DEBUG:teuthology.orchestra.run.smithi098:> /usr/sbin/getenforce 2024-09-10T20:52:14.742 INFO:teuthology.orchestra.run.smithi098.stdout:Permissive 2024-09-10T20:52:14.743 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2024-09-10T20:52:14.764 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2024-09-10T20:52:14.764 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi098.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2024-09-10T20:52:14.764 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T20:52:14.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:52:14.804 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T20:52:14.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:52:14.832 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T20:52:14.832 DEBUG:teuthology.orchestra.run.smithi098:> sudo /usr/sbin/setenforce permissive 2024-09-10T20:52:14.870 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2024-09-10T20:52:14.905 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T20:52:14.914 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T20:52:14.924 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T20:52:14.925 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi098.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T20:55:55.987 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi098.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-09-10T20:55:55.988 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-10T20:55:55.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:55:56.072 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-09-10T20:55:56.152 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-10T20:55:56.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-10T20:55:56.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:55:56.234 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-09-10T20:55:56.321 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-09-10T20:55:56.321 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T20:55:56.331 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T20:55:56.332 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T20:55:56.332 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T20:55:56.334 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T20:55:56.335 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T20:55:56.362 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T20:55:56.377 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T20:55:56.391 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T20:55:56.401 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: ntpd: command not found 2024-09-10T20:55:56.406 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T20:55:56.413 INFO:teuthology.orchestra.run.smithi098.stdout:506 Cannot talk to daemon 2024-09-10T20:55:56.427 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T20:55:56.431 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2024-09-10T20:55:56.441 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T20:55:56.442 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2024-09-10T20:55:56.456 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T20:55:56.470 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T20:55:56.494 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-09-10T20:55:56.498 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T20:55:56.498 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-09-10T20:55:56.498 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:55:56.498 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:55:56.498 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:55:56.498 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:55:56.535 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-09-10T20:55:56.538 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T20:55:56.538 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-09-10T20:55:56.538 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:55:56.538 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:55:56.538 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:55:56.538 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:55:56.539 INFO:teuthology.run_tasks:Running task install... 2024-09-10T20:55:56.550 DEBUG:teuthology.task.install:project ceph 2024-09-10T20:55:56.550 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-10T20:55:56.550 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-10T20:55:56.551 INFO:teuthology.task.install:Using flavor: default 2024-09-10T20:55:56.556 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T20:55:56.556 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T20:55:56.556 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T20:55:56.557 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:55:56.559 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T20:55:56.559 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:55:56.716 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T20:55:56.716 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T20:55:56.727 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T20:55:56.727 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T20:55:56.838 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T20:55:56.838 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:55:56.838 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T20:55:56.846 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T20:55:56.846 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:55:56.846 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T20:55:56.864 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T20:55:56.864 DEBUG:teuthology.orchestra.run.smithi098:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T20:55:56.872 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T20:55:56.872 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T20:55:56.930 DEBUG:teuthology.orchestra.run.smithi098:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T20:55:56.938 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T20:55:57.007 DEBUG:teuthology.orchestra.run.smithi098:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T20:55:57.013 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T20:55:57.072 INFO:teuthology.orchestra.run.smithi098.stdout:check_obsoletes = 1 2024-09-10T20:55:57.073 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum clean all 2024-09-10T20:55:57.078 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2024-09-10T20:55:57.079 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2024-09-10T20:55:57.317 INFO:teuthology.orchestra.run.smithi098.stdout:45 files removed 2024-09-10T20:55:57.322 INFO:teuthology.orchestra.run.smithi190.stdout:45 files removed 2024-09-10T20:55:57.343 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T20:55:57.349 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T20:55:57.911 INFO:teuthology.orchestra.run.smithi098.stdout:ceph packages for x86_64 301 kB/s | 84 kB 00:00 2024-09-10T20:55:57.918 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 295 kB/s | 84 kB 00:00 2024-09-10T20:55:58.194 INFO:teuthology.orchestra.run.smithi098.stdout:ceph noarch packages 101 kB/s | 24 kB 00:00 2024-09-10T20:55:58.218 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-10T20:55:58.470 INFO:teuthology.orchestra.run.smithi098.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-10T20:55:58.494 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-10T20:55:59.027 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-10T20:55:59.087 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-10T20:56:01.113 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-10T20:56:01.162 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-10T20:56:05.381 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-10T20:56:05.412 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-10T20:56:07.005 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 19 kB 00:00 2024-09-10T20:56:07.115 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 19 kB 00:00 2024-09-10T20:56:08.619 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-10T20:56:08.626 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-10T20:56:13.959 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T20:56:14.057 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-10T20:56:16.330 INFO:teuthology.orchestra.run.smithi098.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T20:56:16.331 INFO:teuthology.orchestra.run.smithi098.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T20:56:16.345 INFO:teuthology.orchestra.run.smithi098.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T20:56:16.345 INFO:teuthology.orchestra.run.smithi098.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T20:56:16.346 INFO:teuthology.orchestra.run.smithi098.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T20:56:16.348 INFO:teuthology.orchestra.run.smithi098.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T20:56:16.349 INFO:teuthology.orchestra.run.smithi098.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T20:56:16.352 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T20:56:16.353 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T20:56:16.377 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T20:56:16.378 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T20:56:16.390 INFO:teuthology.orchestra.run.smithi190.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T20:56:16.390 INFO:teuthology.orchestra.run.smithi190.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T20:56:16.391 INFO:teuthology.orchestra.run.smithi190.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T20:56:16.392 INFO:teuthology.orchestra.run.smithi190.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T20:56:16.394 INFO:teuthology.orchestra.run.smithi190.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T20:56:16.396 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T20:56:16.397 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T20:56:16.417 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-10T20:56:16.445 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-09-10T20:56:16.445 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-09-10T20:56:16.445 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-09-10T20:56:16.445 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-10T20:56:16.445 INFO:teuthology.orchestra.run.smithi098.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T20:56:16.445 INFO:teuthology.orchestra.run.smithi098.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T20:56:16.445 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T20:56:16.445 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T20:56:16.445 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T20:56:16.445 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T20:56:16.445 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T20:56:16.445 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T20:56:16.446 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T20:56:16.446 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T20:56:16.446 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T20:56:16.446 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T20:56:16.446 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T20:56:16.446 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T20:56:16.446 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T20:56:16.446 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T20:56:16.446 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T20:56:16.446 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T20:56:16.446 INFO:teuthology.orchestra.run.smithi098.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T20:56:16.446 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T20:56:16.446 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T20:56:16.447 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T20:56:16.447 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T20:56:16.447 INFO:teuthology.orchestra.run.smithi098.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T20:56:16.447 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T20:56:16.447 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T20:56:16.447 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T20:56:16.447 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T20:56:16.447 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T20:56:16.447 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T20:56:16.447 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T20:56:16.447 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T20:56:16.447 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T20:56:16.447 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T20:56:16.447 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T20:56:16.448 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T20:56:16.448 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T20:56:16.448 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrading: 2024-09-10T20:56:16.448 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T20:56:16.448 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T20:56:16.448 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T20:56:16.448 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T20:56:16.448 INFO:teuthology.orchestra.run.smithi098.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T20:56:16.448 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T20:56:16.448 INFO:teuthology.orchestra.run.smithi098.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T20:56:16.448 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T20:56:16.448 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-10T20:56:16.448 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T20:56:16.449 INFO:teuthology.orchestra.run.smithi098.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T20:56:16.449 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T20:56:16.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T20:56:16.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T20:56:16.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T20:56:16.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T20:56:16.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T20:56:16.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T20:56:16.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T20:56:16.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T20:56:16.449 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T20:56:16.449 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T20:56:16.449 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T20:56:16.449 INFO:teuthology.orchestra.run.smithi098.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T20:56:16.450 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T20:56:16.450 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T20:56:16.450 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T20:56:16.450 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T20:56:16.450 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T20:56:16.450 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T20:56:16.450 INFO:teuthology.orchestra.run.smithi098.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T20:56:16.450 INFO:teuthology.orchestra.run.smithi098.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T20:56:16.450 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T20:56:16.450 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T20:56:16.450 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T20:56:16.450 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T20:56:16.450 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T20:56:16.450 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T20:56:16.451 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T20:56:16.451 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T20:56:16.451 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T20:56:16.451 INFO:teuthology.orchestra.run.smithi098.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T20:56:16.451 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T20:56:16.451 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T20:56:16.451 INFO:teuthology.orchestra.run.smithi098.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T20:56:16.451 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T20:56:16.451 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T20:56:16.451 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T20:56:16.451 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T20:56:16.451 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T20:56:16.451 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T20:56:16.451 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T20:56:16.452 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T20:56:16.452 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T20:56:16.452 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T20:56:16.452 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T20:56:16.452 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T20:56:16.452 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T20:56:16.452 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T20:56:16.452 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T20:56:16.452 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T20:56:16.452 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T20:56:16.452 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T20:56:16.452 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T20:56:16.452 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T20:56:16.453 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T20:56:16.453 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T20:56:16.453 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T20:56:16.453 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T20:56:16.453 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T20:56:16.453 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T20:56:16.453 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T20:56:16.453 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T20:56:16.453 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T20:56:16.453 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T20:56:16.453 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T20:56:16.453 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T20:56:16.453 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T20:56:16.453 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T20:56:16.454 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T20:56:16.454 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T20:56:16.454 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T20:56:16.454 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T20:56:16.454 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T20:56:16.454 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T20:56:16.454 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T20:56:16.454 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T20:56:16.454 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T20:56:16.454 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T20:56:16.454 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T20:56:16.454 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T20:56:16.454 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T20:56:16.454 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T20:56:16.455 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T20:56:16.456 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T20:56:16.456 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T20:56:16.456 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T20:56:16.456 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T20:56:16.456 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T20:56:16.456 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T20:56:16.456 INFO:teuthology.orchestra.run.smithi098.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T20:56:16.456 INFO:teuthology.orchestra.run.smithi098.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T20:56:16.456 INFO:teuthology.orchestra.run.smithi098.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T20:56:16.456 INFO:teuthology.orchestra.run.smithi098.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T20:56:16.456 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T20:56:16.456 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T20:56:16.456 INFO:teuthology.orchestra.run.smithi098.stdout:Installing weak dependencies: 2024-09-10T20:56:16.456 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T20:56:16.457 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:56:16.457 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-10T20:56:16.457 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-09-10T20:56:16.457 INFO:teuthology.orchestra.run.smithi098.stdout:Install 149 Packages 2024-09-10T20:56:16.457 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrade 8 Packages 2024-09-10T20:56:16.457 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:56:16.457 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 240 M 2024-09-10T20:56:16.457 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-10T20:56:16.460 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-09-10T20:56:16.475 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-09-10T20:56:16.476 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-09-10T20:56:16.476 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-09-10T20:56:16.476 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-09-10T20:56:16.476 INFO:teuthology.orchestra.run.smithi190.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T20:56:16.476 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T20:56:16.476 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T20:56:16.476 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T20:56:16.476 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T20:56:16.476 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T20:56:16.476 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T20:56:16.476 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T20:56:16.476 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T20:56:16.476 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T20:56:16.477 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T20:56:16.477 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T20:56:16.477 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T20:56:16.477 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T20:56:16.477 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T20:56:16.477 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T20:56:16.477 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T20:56:16.477 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T20:56:16.477 INFO:teuthology.orchestra.run.smithi190.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T20:56:16.477 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T20:56:16.477 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T20:56:16.477 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T20:56:16.477 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T20:56:16.478 INFO:teuthology.orchestra.run.smithi190.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T20:56:16.478 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T20:56:16.478 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T20:56:16.478 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T20:56:16.478 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T20:56:16.478 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T20:56:16.478 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T20:56:16.478 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T20:56:16.478 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T20:56:16.478 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T20:56:16.478 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T20:56:16.478 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T20:56:16.478 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T20:56:16.478 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T20:56:16.479 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2024-09-10T20:56:16.479 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T20:56:16.479 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T20:56:16.479 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T20:56:16.479 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T20:56:16.479 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T20:56:16.479 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T20:56:16.479 INFO:teuthology.orchestra.run.smithi190.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T20:56:16.479 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T20:56:16.479 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-09-10T20:56:16.479 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T20:56:16.479 INFO:teuthology.orchestra.run.smithi190.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T20:56:16.479 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T20:56:16.479 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T20:56:16.480 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T20:56:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T20:56:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T20:56:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T20:56:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T20:56:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T20:56:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T20:56:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T20:56:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T20:56:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T20:56:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T20:56:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T20:56:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T20:56:16.481 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T20:56:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T20:56:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T20:56:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T20:56:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T20:56:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T20:56:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T20:56:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T20:56:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T20:56:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T20:56:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T20:56:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T20:56:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T20:56:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T20:56:16.482 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T20:56:16.483 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T20:56:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T20:56:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T20:56:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T20:56:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T20:56:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T20:56:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T20:56:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T20:56:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T20:56:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T20:56:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T20:56:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T20:56:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T20:56:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T20:56:16.484 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T20:56:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T20:56:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T20:56:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T20:56:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T20:56:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T20:56:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T20:56:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T20:56:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T20:56:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T20:56:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T20:56:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T20:56:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T20:56:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T20:56:16.485 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T20:56:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T20:56:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T20:56:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T20:56:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T20:56:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T20:56:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T20:56:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T20:56:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T20:56:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T20:56:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T20:56:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T20:56:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T20:56:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T20:56:16.486 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T20:56:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T20:56:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T20:56:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T20:56:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T20:56:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T20:56:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T20:56:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T20:56:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T20:56:16.487 INFO:teuthology.orchestra.run.smithi190.stdout:Installing weak dependencies: 2024-09-10T20:56:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T20:56:16.487 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:56:16.487 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-09-10T20:56:16.487 INFO:teuthology.orchestra.run.smithi190.stdout:======================================================================================= 2024-09-10T20:56:16.487 INFO:teuthology.orchestra.run.smithi190.stdout:Install 149 Packages 2024-09-10T20:56:16.488 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 8 Packages 2024-09-10T20:56:16.488 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:56:16.488 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 240 M 2024-09-10T20:56:16.488 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-09-10T20:56:17.377 INFO:teuthology.orchestra.run.smithi098.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-09-10T20:56:17.841 INFO:teuthology.orchestra.run.smithi190.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 12 kB/s | 6.6 kB 00:00 2024-09-10T20:56:17.852 INFO:teuthology.orchestra.run.smithi098.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 1.9 MB/s | 900 kB 00:00 2024-09-10T20:56:17.911 INFO:teuthology.orchestra.run.smithi098.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 2.4 MB/s | 145 kB 00:00 2024-09-10T20:56:18.103 INFO:teuthology.orchestra.run.smithi098.stdout:(4/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 12 MB/s | 2.4 MB 00:00 2024-09-10T20:56:18.253 INFO:teuthology.orchestra.run.smithi098.stdout:(5/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 10 MB/s | 1.5 MB 00:00 2024-09-10T20:56:18.325 INFO:teuthology.orchestra.run.smithi190.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 1.8 MB/s | 900 kB 00:00 2024-09-10T20:56:18.416 INFO:teuthology.orchestra.run.smithi190.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.5 MB/s | 145 kB 00:00 2024-09-10T20:56:18.580 INFO:teuthology.orchestra.run.smithi098.stdout:(6/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 15 MB/s | 4.8 MB 00:00 2024-09-10T20:56:19.225 INFO:teuthology.orchestra.run.smithi190.stdout:(4/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 2.9 MB/s | 2.4 MB 00:00 2024-09-10T20:56:19.554 INFO:teuthology.orchestra.run.smithi098.stdout:(7/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x86 18 MB/s | 17 MB 00:00 2024-09-10T20:56:19.750 INFO:teuthology.orchestra.run.smithi190.stdout:(5/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 2.9 MB/s | 1.5 MB 00:00 2024-09-10T20:56:20.262 INFO:teuthology.orchestra.run.smithi098.stdout:(8/157): ceph-radosgw-19.3.0-4854.g8f219c47.el9 15 MB/s | 11 MB 00:00 2024-09-10T20:56:20.379 INFO:teuthology.orchestra.run.smithi098.stdout:(9/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 1.6 MB/s | 5.5 MB 00:03 2024-09-10T20:56:20.405 INFO:teuthology.orchestra.run.smithi098.stdout:(10/157): ceph-selinux-19.3.0-4854.g8f219c47.el 177 kB/s | 25 kB 00:00 2024-09-10T20:56:20.480 INFO:teuthology.orchestra.run.smithi098.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 2.9 MB/s | 225 kB 00:00 2024-09-10T20:56:20.555 INFO:teuthology.orchestra.run.smithi098.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 360 kB/s | 27 kB 00:00 2024-09-10T20:56:20.624 INFO:teuthology.orchestra.run.smithi098.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 477 kB/s | 32 kB 00:00 2024-09-10T20:56:20.714 INFO:teuthology.orchestra.run.smithi098.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 8.0 MB/s | 755 kB 00:00 2024-09-10T20:56:20.790 INFO:teuthology.orchestra.run.smithi098.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.1 MB/s | 162 kB 00:00 2024-09-10T20:56:20.881 INFO:teuthology.orchestra.run.smithi098.stdout:(16/157): librados-devel-19.3.0-4854.g8f219c47. 1.4 MB/s | 127 kB 00:00 2024-09-10T20:56:20.973 INFO:teuthology.orchestra.run.smithi098.stdout:(17/157): libradosstriper1-19.3.0-4854.g8f219c4 5.4 MB/s | 507 kB 00:00 2024-09-10T20:56:21.159 INFO:teuthology.orchestra.run.smithi190.stdout:(6/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 3.4 MB/s | 4.8 MB 00:01 2024-09-10T20:56:21.316 INFO:teuthology.orchestra.run.smithi098.stdout:(18/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 16 MB/s | 5.5 MB 00:00 2024-09-10T20:56:21.390 INFO:teuthology.orchestra.run.smithi098.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 605 kB/s | 45 kB 00:00 2024-09-10T20:56:21.466 INFO:teuthology.orchestra.run.smithi098.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.9 MB/s | 143 kB 00:00 2024-09-10T20:56:21.541 INFO:teuthology.orchestra.run.smithi098.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 2.1 MB/s | 161 kB 00:00 2024-09-10T20:56:21.617 INFO:teuthology.orchestra.run.smithi098.stdout:(22/157): python3-rados-19.3.0-4854.g8f219c47.e 4.2 MB/s | 321 kB 00:00 2024-09-10T20:56:21.676 INFO:teuthology.orchestra.run.smithi190.stdout:(7/157): ceph-common-19.3.0-4854.g8f219c47.el9. 4.9 MB/s | 22 MB 00:04 2024-09-10T20:56:21.692 INFO:teuthology.orchestra.run.smithi098.stdout:(23/157): python3-rbd-19.3.0-4854.g8f219c47.el9 4.0 MB/s | 306 kB 00:00 2024-09-10T20:56:21.751 INFO:teuthology.orchestra.run.smithi098.stdout:(24/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.7 MB/s | 99 kB 00:00 2024-09-10T20:56:21.809 INFO:teuthology.orchestra.run.smithi098.stdout:(25/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.4 MB/s | 85 kB 00:00 2024-09-10T20:56:22.026 INFO:teuthology.orchestra.run.smithi098.stdout:(26/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 15 MB/s | 3.2 MB 00:00 2024-09-10T20:56:22.093 INFO:teuthology.orchestra.run.smithi098.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.5 MB/s | 171 kB 00:00 2024-09-10T20:56:22.152 INFO:teuthology.orchestra.run.smithi098.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 586 kB/s | 34 kB 00:00 2024-09-10T20:56:22.236 INFO:teuthology.orchestra.run.smithi098.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 1.8 MB/s | 154 kB 00:00 2024-09-10T20:56:23.103 INFO:teuthology.orchestra.run.smithi098.stdout:(30/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 16 MB/s | 14 MB 00:00 2024-09-10T20:56:23.384 INFO:teuthology.orchestra.run.smithi190.stdout:(8/157): ceph-radosgw-19.3.0-4854.g8f219c47.el9 6.4 MB/s | 11 MB 00:01 2024-09-10T20:56:23.419 INFO:teuthology.orchestra.run.smithi098.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 23 MB/s | 7.4 MB 00:00 2024-09-10T20:56:23.470 INFO:teuthology.orchestra.run.smithi098.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 5.7 MB/s | 291 kB 00:00 2024-09-10T20:56:23.476 INFO:teuthology.orchestra.run.smithi190.stdout:(9/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 909 kB/s | 5.5 MB 00:06 2024-09-10T20:56:23.501 INFO:teuthology.orchestra.run.smithi190.stdout:(10/157): ceph-selinux-19.3.0-4854.g8f219c47.el 215 kB/s | 25 kB 00:00 2024-09-10T20:56:23.520 INFO:teuthology.orchestra.run.smithi098.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 991 kB/s | 49 kB 00:00 2024-09-10T20:56:23.562 INFO:teuthology.orchestra.run.smithi098.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 394 kB/s | 16 kB 00:00 2024-09-10T20:56:23.568 INFO:teuthology.orchestra.run.smithi190.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 3.3 MB/s | 225 kB 00:00 2024-09-10T20:56:23.610 INFO:teuthology.orchestra.run.smithi190.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 651 kB/s | 27 kB 00:00 2024-09-10T20:56:23.612 INFO:teuthology.orchestra.run.smithi098.stdout:(35/157): ceph-volume-19.3.0-4854.g8f219c47.el9 5.6 MB/s | 285 kB 00:00 2024-09-10T20:56:23.652 INFO:teuthology.orchestra.run.smithi190.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 763 kB/s | 32 kB 00:00 2024-09-10T20:56:23.679 INFO:teuthology.orchestra.run.smithi098.stdout:(36/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 11 MB/s | 783 kB 00:00 2024-09-10T20:56:23.721 INFO:teuthology.orchestra.run.smithi098.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 439 kB/s | 18 kB 00:00 2024-09-10T20:56:23.752 INFO:teuthology.orchestra.run.smithi190.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 7.4 MB/s | 755 kB 00:00 2024-09-10T20:56:23.788 INFO:teuthology.orchestra.run.smithi098.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 1.2 MB/s | 80 kB 00:00 2024-09-10T20:56:23.811 INFO:teuthology.orchestra.run.smithi190.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.7 MB/s | 162 kB 00:00 2024-09-10T20:56:23.861 INFO:teuthology.orchestra.run.smithi190.stdout:(16/157): librados-devel-19.3.0-4854.g8f219c47. 2.5 MB/s | 127 kB 00:00 2024-09-10T20:56:23.889 INFO:teuthology.orchestra.run.smithi098.stdout:(39/157): libgfortran-11.5.0-2.el9.x86_64.rpm 7.8 MB/s | 801 kB 00:00 2024-09-10T20:56:23.928 INFO:teuthology.orchestra.run.smithi190.stdout:(17/157): libradosstriper1-19.3.0-4854.g8f219c4 7.5 MB/s | 507 kB 00:00 2024-09-10T20:56:23.972 INFO:teuthology.orchestra.run.smithi098.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-10T20:56:24.056 INFO:teuthology.orchestra.run.smithi098.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-10T20:56:24.164 INFO:teuthology.orchestra.run.smithi098.stdout:(42/157): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-10T20:56:24.256 INFO:teuthology.orchestra.run.smithi098.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-09-10T20:56:24.332 INFO:teuthology.orchestra.run.smithi098.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-10T20:56:24.415 INFO:teuthology.orchestra.run.smithi098.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-10T20:56:24.422 INFO:teuthology.orchestra.run.smithi190.stdout:(18/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 11 MB/s | 5.5 MB 00:00 2024-09-10T20:56:24.472 INFO:teuthology.orchestra.run.smithi190.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 909 kB/s | 45 kB 00:00 2024-09-10T20:56:24.507 INFO:teuthology.orchestra.run.smithi098.stdout:(46/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-10T20:56:24.522 INFO:teuthology.orchestra.run.smithi190.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 2.8 MB/s | 143 kB 00:00 2024-09-10T20:56:24.681 INFO:teuthology.orchestra.run.smithi190.stdout:(21/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 4.9 MB/s | 17 MB 00:03 2024-09-10T20:56:24.708 INFO:teuthology.orchestra.run.smithi190.stdout:(22/157): python3-cephfs-19.3.0-4854.g8f219c47. 880 kB/s | 161 kB 00:00 2024-09-10T20:56:24.765 INFO:teuthology.orchestra.run.smithi190.stdout:(23/157): python3-rados-19.3.0-4854.g8f219c47.e 3.8 MB/s | 321 kB 00:00 2024-09-10T20:56:24.790 INFO:teuthology.orchestra.run.smithi190.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 3.6 MB/s | 306 kB 00:00 2024-09-10T20:56:24.816 INFO:teuthology.orchestra.run.smithi190.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.9 MB/s | 99 kB 00:00 2024-09-10T20:56:24.833 INFO:teuthology.orchestra.run.smithi098.stdout:(47/157): ceph-common-19.3.0-4854.g8f219c47.el9 2.8 MB/s | 22 MB 00:07 2024-09-10T20:56:24.841 INFO:teuthology.orchestra.run.smithi190.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.7 MB/s | 85 kB 00:00 2024-09-10T20:56:24.891 INFO:teuthology.orchestra.run.smithi190.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 3.3 MB/s | 171 kB 00:00 2024-09-10T20:56:24.941 INFO:teuthology.orchestra.run.smithi190.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 686 kB/s | 34 kB 00:00 2024-09-10T20:56:24.992 INFO:teuthology.orchestra.run.smithi190.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 3.0 MB/s | 154 kB 00:00 2024-09-10T20:56:25.234 INFO:teuthology.orchestra.run.smithi190.stdout:(30/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 7.6 MB/s | 3.2 MB 00:00 2024-09-10T20:56:25.417 INFO:teuthology.orchestra.run.smithi098.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 758 kB/s | 689 kB 00:00 2024-09-10T20:56:25.542 INFO:teuthology.orchestra.run.smithi098.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 856 kB/s | 107 kB 00:00 2024-09-10T20:56:25.667 INFO:teuthology.orchestra.run.smithi098.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-10T20:56:25.692 INFO:teuthology.orchestra.run.smithi098.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 144 kB/s | 22 kB 00:00 2024-09-10T20:56:25.784 INFO:teuthology.orchestra.run.smithi098.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-10T20:56:25.801 INFO:teuthology.orchestra.run.smithi098.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-10T20:56:25.910 INFO:teuthology.orchestra.run.smithi098.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-10T20:56:26.027 INFO:teuthology.orchestra.run.smithi098.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-09-10T20:56:26.152 INFO:teuthology.orchestra.run.smithi098.stdout:(56/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.3 MB/s | 171 kB 00:00 2024-09-10T20:56:26.217 INFO:teuthology.orchestra.run.smithi190.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 7.5 MB/s | 7.4 MB 00:00 2024-09-10T20:56:26.269 INFO:teuthology.orchestra.run.smithi098.stdout:(57/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-10T20:56:26.320 INFO:teuthology.orchestra.run.smithi098.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.6 MB/s | 3.0 MB 00:00 2024-09-10T20:56:26.351 INFO:teuthology.orchestra.run.smithi190.stdout:(32/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 10 MB/s | 14 MB 00:01 2024-09-10T20:56:26.370 INFO:teuthology.orchestra.run.smithi098.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-09-10T20:56:26.376 INFO:teuthology.orchestra.run.smithi190.stdout:(33/157): ceph-mgr-modules-core-19.3.0-4854.g8f 1.8 MB/s | 291 kB 00:00 2024-09-10T20:56:26.401 INFO:teuthology.orchestra.run.smithi190.stdout:(34/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 987 kB/s | 49 kB 00:00 2024-09-10T20:56:26.427 INFO:teuthology.orchestra.run.smithi190.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4854.g8 326 kB/s | 16 kB 00:00 2024-09-10T20:56:26.452 INFO:teuthology.orchestra.run.smithi190.stdout:(36/157): ceph-volume-19.3.0-4854.g8f219c47.el9 5.5 MB/s | 285 kB 00:00 2024-09-10T20:56:26.495 INFO:teuthology.orchestra.run.smithi098.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-10T20:56:26.502 INFO:teuthology.orchestra.run.smithi190.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 364 kB/s | 18 kB 00:00 2024-09-10T20:56:26.553 INFO:teuthology.orchestra.run.smithi190.stdout:(38/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 6.1 MB/s | 783 kB 00:00 2024-09-10T20:56:26.595 INFO:teuthology.orchestra.run.smithi098.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 245 kB/s | 24 kB 00:00 2024-09-10T20:56:26.670 INFO:teuthology.orchestra.run.smithi190.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-10T20:56:26.696 INFO:teuthology.orchestra.run.smithi098.stdout:(62/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 475 kB/s | 47 kB 00:00 2024-09-10T20:56:26.756 INFO:teuthology.orchestra.run.smithi190.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-10T20:56:26.813 INFO:teuthology.orchestra.run.smithi098.stdout:(63/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-10T20:56:26.837 INFO:teuthology.orchestra.run.smithi190.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-10T20:56:26.862 INFO:teuthology.orchestra.run.smithi190.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-10T20:56:26.930 INFO:teuthology.orchestra.run.smithi098.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-10T20:56:27.029 INFO:teuthology.orchestra.run.smithi190.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 639 kB/s | 106 kB 00:00 2024-09-10T20:56:27.038 INFO:teuthology.orchestra.run.smithi098.stdout:(65/157): openblas-0.3.26-2.el9.x86_64.rpm 356 kB/s | 38 kB 00:00 2024-09-10T20:56:27.096 INFO:teuthology.orchestra.run.smithi190.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 4.7 MB/s | 1.2 MB 00:00 2024-09-10T20:56:27.130 INFO:teuthology.orchestra.run.smithi190.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-10T20:56:27.164 INFO:teuthology.orchestra.run.smithi190.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-10T20:56:27.214 INFO:teuthology.orchestra.run.smithi190.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-10T20:56:27.330 INFO:teuthology.orchestra.run.smithi098.stdout:(66/157): librdkafka-1.6.1-102.el9.x86_64.rpm 656 kB/s | 662 kB 00:01 2024-09-10T20:56:27.448 INFO:teuthology.orchestra.run.smithi190.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-10T20:56:27.523 INFO:teuthology.orchestra.run.smithi190.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-09-10T20:56:27.548 INFO:teuthology.orchestra.run.smithi190.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-09-10T20:56:27.582 INFO:teuthology.orchestra.run.smithi190.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 370 kB/s | 22 kB 00:00 2024-09-10T20:56:27.616 INFO:teuthology.orchestra.run.smithi190.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-09-10T20:56:27.641 INFO:teuthology.orchestra.run.smithi190.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-10T20:56:27.691 INFO:teuthology.orchestra.run.smithi190.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-10T20:56:27.750 INFO:teuthology.orchestra.run.smithi190.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-09-10T20:56:27.800 INFO:teuthology.orchestra.run.smithi190.stdout:(56/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-09-10T20:56:27.825 INFO:teuthology.orchestra.run.smithi190.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.2 MB/s | 171 kB 00:00 2024-09-10T20:56:27.859 INFO:teuthology.orchestra.run.smithi190.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-10T20:56:27.884 INFO:teuthology.orchestra.run.smithi190.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 770 kB/s | 45 kB 00:00 2024-09-10T20:56:27.918 INFO:teuthology.orchestra.run.smithi190.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-10T20:56:27.960 INFO:teuthology.orchestra.run.smithi190.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-10T20:56:27.985 INFO:teuthology.orchestra.run.smithi190.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 365 kB/s | 24 kB 00:00 2024-09-10T20:56:28.011 INFO:teuthology.orchestra.run.smithi190.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-10T20:56:28.052 INFO:teuthology.orchestra.run.smithi190.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-09-10T20:56:28.078 INFO:teuthology.orchestra.run.smithi190.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-09-10T20:56:28.112 INFO:teuthology.orchestra.run.smithi190.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 657 kB/s | 38 kB 00:00 2024-09-10T20:56:28.295 INFO:teuthology.orchestra.run.smithi190.stdout:(67/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-09-10T20:56:28.362 INFO:teuthology.orchestra.run.smithi190.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 17 MB/s | 4.9 MB 00:00 2024-09-10T20:56:28.404 INFO:teuthology.orchestra.run.smithi190.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-10T20:56:28.438 INFO:teuthology.orchestra.run.smithi190.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-10T20:56:28.463 INFO:teuthology.orchestra.run.smithi190.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-10T20:56:28.564 INFO:teuthology.orchestra.run.smithi190.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-10T20:56:28.639 INFO:teuthology.orchestra.run.smithi190.stdout:(73/157): postgresql-server-13.16-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-09-10T20:56:28.664 INFO:teuthology.orchestra.run.smithi190.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 871 kB/s | 87 kB 00:00 2024-09-10T20:56:28.748 INFO:teuthology.orchestra.run.smithi190.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 735 kB/s | 61 kB 00:00 2024-09-10T20:56:28.823 INFO:teuthology.orchestra.run.smithi190.stdout:(76/157): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2024-09-10T20:56:28.857 INFO:teuthology.orchestra.run.smithi190.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-10T20:56:28.882 INFO:teuthology.orchestra.run.smithi190.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-10T20:56:28.907 INFO:teuthology.orchestra.run.smithi190.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 508 kB/s | 26 kB 00:00 2024-09-10T20:56:28.933 INFO:teuthology.orchestra.run.smithi190.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-10T20:56:28.947 INFO:teuthology.orchestra.run.smithi098.stdout:(67/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.7 MB/s | 4.4 MB 00:01 2024-09-10T20:56:28.966 INFO:teuthology.orchestra.run.smithi190.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-10T20:56:28.983 INFO:teuthology.orchestra.run.smithi190.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-10T20:56:29.059 INFO:teuthology.orchestra.run.smithi190.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-09-10T20:56:29.126 INFO:teuthology.orchestra.run.smithi190.stdout:(84/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-09-10T20:56:29.151 INFO:teuthology.orchestra.run.smithi190.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-10T20:56:29.193 INFO:teuthology.orchestra.run.smithi190.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-10T20:56:29.218 INFO:teuthology.orchestra.run.smithi190.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 632 kB/s | 42 kB 00:00 2024-09-10T20:56:29.252 INFO:teuthology.orchestra.run.smithi190.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-10T20:56:29.285 INFO:teuthology.orchestra.run.smithi190.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-10T20:56:29.311 INFO:teuthology.orchestra.run.smithi190.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-10T20:56:29.336 INFO:teuthology.orchestra.run.smithi190.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T20:56:29.378 INFO:teuthology.orchestra.run.smithi190.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 804 kB/s | 54 kB 00:00 2024-09-10T20:56:29.478 INFO:teuthology.orchestra.run.smithi190.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-09-10T20:56:29.537 INFO:teuthology.orchestra.run.smithi190.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-09-10T20:56:29.572 INFO:teuthology.orchestra.run.smithi098.stdout:(68/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 5.5 MB/s | 50 MB 00:09 2024-09-10T20:56:29.614 INFO:teuthology.orchestra.run.smithi098.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 2.4 MB/s | 1.6 MB 00:00 2024-09-10T20:56:29.620 INFO:teuthology.orchestra.run.smithi190.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-10T20:56:29.679 INFO:teuthology.orchestra.run.smithi190.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-09-10T20:56:29.731 INFO:teuthology.orchestra.run.smithi098.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-10T20:56:29.838 INFO:teuthology.orchestra.run.smithi190.stdout:(97/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-10T20:56:29.854 INFO:teuthology.orchestra.run.smithi190.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 270 kB/s | 47 kB 00:00 2024-09-10T20:56:29.914 INFO:teuthology.orchestra.run.smithi190.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 3.9 MB/s | 304 kB 00:00 2024-09-10T20:56:29.939 INFO:teuthology.orchestra.run.smithi190.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 688 kB/s | 58 kB 00:00 2024-09-10T20:56:29.973 INFO:teuthology.orchestra.run.smithi190.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-10T20:56:29.998 INFO:teuthology.orchestra.run.smithi190.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-10T20:56:30.031 INFO:teuthology.orchestra.run.smithi190.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 763 kB/s | 45 kB 00:00 2024-09-10T20:56:30.099 INFO:teuthology.orchestra.run.smithi190.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-10T20:56:30.174 INFO:teuthology.orchestra.run.smithi190.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 7.2 MB/s | 548 kB 00:00 2024-09-10T20:56:30.207 INFO:teuthology.orchestra.run.smithi190.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 9.0 MB/s | 309 kB 00:00 2024-09-10T20:56:30.233 INFO:teuthology.orchestra.run.smithi190.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-10T20:56:30.416 INFO:teuthology.orchestra.run.smithi190.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-09-10T20:56:30.465 INFO:teuthology.orchestra.run.smithi098.stdout:(71/157): postgresql-contrib-13.16-1.el9.x86_64 1.0 MB/s | 910 kB 00:00 2024-09-10T20:56:30.467 INFO:teuthology.orchestra.run.smithi190.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-09-10T20:56:30.532 INFO:teuthology.orchestra.run.smithi098.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 1.4 MB/s | 4.9 MB 00:03 2024-09-10T20:56:30.534 INFO:teuthology.orchestra.run.smithi190.stdout:(110/157): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-10T20:56:30.559 INFO:teuthology.orchestra.run.smithi190.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-09-10T20:56:30.592 INFO:teuthology.orchestra.run.smithi190.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-10T20:56:30.626 INFO:teuthology.orchestra.run.smithi190.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-10T20:56:30.685 INFO:teuthology.orchestra.run.smithi190.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-10T20:56:30.719 INFO:teuthology.orchestra.run.smithi190.stdout:(115/157): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-09-10T20:56:30.744 INFO:teuthology.orchestra.run.smithi190.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 743 kB/s | 43 kB 00:00 2024-09-10T20:56:30.749 INFO:teuthology.orchestra.run.smithi098.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 404 kB/s | 87 kB 00:00 2024-09-10T20:56:30.769 INFO:teuthology.orchestra.run.smithi190.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-09-10T20:56:30.782 INFO:teuthology.orchestra.run.smithi098.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 3.2 MB/s | 1.0 MB 00:00 2024-09-10T20:56:30.794 INFO:teuthology.orchestra.run.smithi190.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-10T20:56:30.828 INFO:teuthology.orchestra.run.smithi190.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-10T20:56:30.878 INFO:teuthology.orchestra.run.smithi190.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-10T20:56:30.899 INFO:teuthology.orchestra.run.smithi098.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 525 kB/s | 61 kB 00:00 2024-09-10T20:56:30.920 INFO:teuthology.orchestra.run.smithi190.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-09-10T20:56:30.945 INFO:teuthology.orchestra.run.smithi190.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-10T20:56:30.971 INFO:teuthology.orchestra.run.smithi190.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-10T20:56:30.996 INFO:teuthology.orchestra.run.smithi190.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-10T20:56:31.022 INFO:teuthology.orchestra.run.smithi190.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 696 kB/s | 18 kB 00:00 2024-09-10T20:56:31.024 INFO:teuthology.orchestra.run.smithi098.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-10T20:56:31.047 INFO:teuthology.orchestra.run.smithi190.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-10T20:56:31.072 INFO:teuthology.orchestra.run.smithi190.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 773 kB/s | 19 kB 00:00 2024-09-10T20:56:31.098 INFO:teuthology.orchestra.run.smithi190.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 786 kB/s | 20 kB 00:00 2024-09-10T20:56:31.141 INFO:teuthology.orchestra.run.smithi098.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-09-10T20:56:31.156 INFO:teuthology.orchestra.run.smithi190.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.6 MB/s | 2.0 MB 00:00 2024-09-10T20:56:31.173 INFO:teuthology.orchestra.run.smithi190.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.8 MB/s | 46 kB 00:00 2024-09-10T20:56:31.250 INFO:teuthology.orchestra.run.smithi098.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-09-10T20:56:31.384 INFO:teuthology.orchestra.run.smithi098.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 144 kB/s | 19 kB 00:00 2024-09-10T20:56:31.518 INFO:teuthology.orchestra.run.smithi098.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-10T20:56:31.618 INFO:teuthology.orchestra.run.smithi098.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 348 kB/s | 35 kB 00:00 2024-09-10T20:56:31.632 INFO:teuthology.orchestra.run.smithi190.stdout:(131/157): ceph-test-19.3.0-4854.g8f219c47.el9. 6.2 MB/s | 50 MB 00:08 2024-09-10T20:56:31.657 INFO:teuthology.orchestra.run.smithi190.stdout:(132/157): python3-more-itertools-8.12.0-2.el9. 163 kB/s | 79 kB 00:00 2024-09-10T20:56:31.707 INFO:teuthology.orchestra.run.smithi190.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 1.7 MB/s | 1.0 MB 00:00 2024-09-10T20:56:31.733 INFO:teuthology.orchestra.run.smithi190.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 708 kB/s | 16 kB 00:00 2024-09-10T20:56:31.758 INFO:teuthology.orchestra.run.smithi190.stdout:(135/157): python3-natsort-7.1.1-5.el9.noarch.r 460 kB/s | 58 kB 00:00 2024-09-10T20:56:31.792 INFO:teuthology.orchestra.run.smithi190.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-10T20:56:31.817 INFO:teuthology.orchestra.run.smithi190.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-10T20:56:31.842 INFO:teuthology.orchestra.run.smithi190.stdout:(138/157): python3-pyperclip-1.8.2-2.el9.noarch 337 kB/s | 28 kB 00:00 2024-09-10T20:56:31.868 INFO:teuthology.orchestra.run.smithi190.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-09-10T20:56:31.893 INFO:teuthology.orchestra.run.smithi190.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-10T20:56:31.918 INFO:teuthology.orchestra.run.smithi190.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 716 kB/s | 36 kB 00:00 2024-09-10T20:56:31.952 INFO:teuthology.orchestra.run.smithi190.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-10T20:56:31.977 INFO:teuthology.orchestra.run.smithi190.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 891 kB/s | 74 kB 00:00 2024-09-10T20:56:32.002 INFO:teuthology.orchestra.run.smithi190.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-10T20:56:32.028 INFO:teuthology.orchestra.run.smithi190.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-10T20:56:32.053 INFO:teuthology.orchestra.run.smithi190.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-10T20:56:32.078 INFO:teuthology.orchestra.run.smithi190.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T20:56:32.112 INFO:teuthology.orchestra.run.smithi190.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-10T20:56:32.229 INFO:teuthology.orchestra.run.smithi190.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-09-10T20:56:32.335 INFO:teuthology.orchestra.run.smithi098.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 2.2 MB/s | 5.8 MB 00:02 2024-09-10T20:56:32.462 INFO:teuthology.orchestra.run.smithi190.stdout:(150/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 9.3 MB/s | 3.2 MB 00:00 2024-09-10T20:56:32.596 INFO:teuthology.orchestra.run.smithi190.stdout:(151/157): librados2-19.3.0-4854.g8f219c47.el9. 6.7 MB/s | 3.5 MB 00:00 2024-09-10T20:56:32.727 INFO:teuthology.orchestra.run.smithi098.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 640 kB/s | 250 kB 00:00 2024-09-10T20:56:32.738 INFO:teuthology.orchestra.run.smithi190.stdout:(152/157): NetworkManager-1.49.90-1.el9.x86_64. 4.6 MB/s | 2.3 MB 00:00 2024-09-10T20:56:32.764 INFO:teuthology.orchestra.run.smithi190.stdout:(153/157): NetworkManager-team-1.49.90-1.el9.x8 243 kB/s | 41 kB 00:00 2024-09-10T20:56:32.831 INFO:teuthology.orchestra.run.smithi190.stdout:(154/157): NetworkManager-tui-1.49.90-1.el9.x86 2.6 MB/s | 249 kB 00:00 2024-09-10T20:56:32.964 INFO:teuthology.orchestra.run.smithi190.stdout:(155/157): NetworkManager-libnm-1.49.90-1.el9.x 3.6 MB/s | 1.8 MB 00:00 2024-09-10T20:56:33.089 INFO:teuthology.orchestra.run.smithi190.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 4.2 MB/s | 1.4 MB 00:00 2024-09-10T20:56:33.174 INFO:teuthology.orchestra.run.smithi098.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 498 kB/s | 222 kB 00:00 2024-09-10T20:56:33.273 INFO:teuthology.orchestra.run.smithi190.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 5.4 MB/s | 2.4 MB 00:00 2024-09-10T20:56:33.279 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:56:33.280 INFO:teuthology.orchestra.run.smithi190.stdout:Total 14 MB/s | 240 MB 00:16 2024-09-10T20:56:33.377 INFO:teuthology.orchestra.run.smithi098.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 379 kB/s | 77 kB 00:00 2024-09-10T20:56:33.478 INFO:teuthology.orchestra.run.smithi098.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 423 kB/s | 42 kB 00:00 2024-09-10T20:56:33.945 INFO:teuthology.orchestra.run.smithi098.stdout:(87/157): python3-protobuf-3.14.0-13.el9.noarch 577 kB/s | 269 kB 00:00 2024-09-10T20:56:34.237 INFO:teuthology.orchestra.run.smithi098.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 544 kB/s | 159 kB 00:00 2024-09-10T20:56:34.503 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-09-10T20:56:34.570 INFO:teuthology.orchestra.run.smithi098.stdout:(89/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.7 MB/s | 5.1 MB 00:02 2024-09-10T20:56:34.654 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-09-10T20:56:34.654 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-09-10T20:56:34.678 INFO:teuthology.orchestra.run.smithi098.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-09-10T20:56:34.754 INFO:teuthology.orchestra.run.smithi098.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 539 kB/s | 279 kB 00:00 2024-09-10T20:56:34.779 INFO:teuthology.orchestra.run.smithi098.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 534 kB/s | 54 kB 00:00 2024-09-10T20:56:34.888 INFO:teuthology.orchestra.run.smithi098.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-09-10T20:56:34.988 INFO:teuthology.orchestra.run.smithi098.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-09-10T20:56:35.397 INFO:teuthology.orchestra.run.smithi098.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 540 kB/s | 221 kB 00:00 2024-09-10T20:56:35.506 INFO:teuthology.orchestra.run.smithi098.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 332 kB/s | 36 kB 00:00 2024-09-10T20:56:35.706 INFO:teuthology.orchestra.run.smithi098.stdout:(97/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-09-10T20:56:35.731 INFO:teuthology.orchestra.run.smithi098.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 210 kB/s | 47 kB 00:00 2024-09-10T20:56:35.824 INFO:teuthology.orchestra.run.smithi098.stdout:(99/157): python3-babel-2.9.1-2.el9.noarch.rpm 1.2 MB/s | 6.0 MB 00:05 2024-09-10T20:56:35.849 INFO:teuthology.orchestra.run.smithi098.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 494 kB/s | 58 kB 00:00 2024-09-10T20:56:35.882 INFO:teuthology.orchestra.run.smithi098.stdout:(101/157): socat-1.7.4.1-6.el9.x86_64.rpm 1.7 MB/s | 304 kB 00:00 2024-09-10T20:56:35.932 INFO:teuthology.orchestra.run.smithi098.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 881 kB/s | 96 kB 00:00 2024-09-10T20:56:35.958 INFO:teuthology.orchestra.run.smithi098.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 585 kB/s | 64 kB 00:00 2024-09-10T20:56:36.000 INFO:teuthology.orchestra.run.smithi098.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 382 kB/s | 45 kB 00:00 2024-09-10T20:56:36.083 INFO:teuthology.orchestra.run.smithi098.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-10T20:56:36.125 INFO:teuthology.orchestra.run.smithi098.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-09-10T20:56:36.151 INFO:teuthology.orchestra.run.smithi098.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 773 kB/s | 19 kB 00:00 2024-09-10T20:56:36.468 INFO:teuthology.orchestra.run.smithi098.stdout:(108/157): lua-devel-5.4.4-4.el9.x86_64.rpm 42 kB/s | 22 kB 00:00 2024-09-10T20:56:36.509 INFO:teuthology.orchestra.run.smithi098.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 631 kB/s | 26 kB 00:00 2024-09-10T20:56:36.535 INFO:teuthology.orchestra.run.smithi098.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T20:56:36.602 INFO:teuthology.orchestra.run.smithi098.stdout:(111/157): libarrow-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-09-10T20:56:36.627 INFO:teuthology.orchestra.run.smithi098.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 733 kB/s | 67 kB 00:00 2024-09-10T20:56:36.653 INFO:teuthology.orchestra.run.smithi098.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-10T20:56:36.753 INFO:teuthology.orchestra.run.smithi098.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-09-10T20:56:36.778 INFO:teuthology.orchestra.run.smithi098.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T20:56:36.803 INFO:teuthology.orchestra.run.smithi098.stdout:(116/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-09-10T20:56:36.829 INFO:teuthology.orchestra.run.smithi098.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-09-10T20:56:36.854 INFO:teuthology.orchestra.run.smithi098.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-10T20:56:36.896 INFO:teuthology.orchestra.run.smithi098.stdout:(119/157): protobuf-compiler-3.14.0-13.el9.x86_ 920 kB/s | 863 kB 00:00 2024-09-10T20:56:36.921 INFO:teuthology.orchestra.run.smithi098.stdout:(120/157): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-09-10T20:56:36.950 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-09-10T20:56:36.950 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-09-10T20:56:36.955 INFO:teuthology.orchestra.run.smithi098.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 8.5 MB/s | 220 kB 00:00 2024-09-10T20:56:36.980 INFO:teuthology.orchestra.run.smithi098.stdout:(122/157): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-09-10T20:56:37.014 INFO:teuthology.orchestra.run.smithi098.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T20:56:37.039 INFO:teuthology.orchestra.run.smithi098.stdout:(124/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-09-10T20:56:37.064 INFO:teuthology.orchestra.run.smithi098.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-09-10T20:56:37.090 INFO:teuthology.orchestra.run.smithi098.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-10T20:56:37.115 INFO:teuthology.orchestra.run.smithi098.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-10T20:56:37.141 INFO:teuthology.orchestra.run.smithi098.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-10T20:56:37.166 INFO:teuthology.orchestra.run.smithi098.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-10T20:56:37.191 INFO:teuthology.orchestra.run.smithi098.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T20:56:37.225 INFO:teuthology.orchestra.run.smithi098.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-10T20:56:37.259 INFO:teuthology.orchestra.run.smithi098.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-10T20:56:37.309 INFO:teuthology.orchestra.run.smithi098.stdout:(133/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-09-10T20:56:37.342 INFO:teuthology.orchestra.run.smithi098.stdout:(134/157): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-09-10T20:56:37.368 INFO:teuthology.orchestra.run.smithi098.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 279 kB/s | 16 kB 00:00 2024-09-10T20:56:37.393 INFO:teuthology.orchestra.run.smithi098.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-10T20:56:37.419 INFO:teuthology.orchestra.run.smithi098.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-10T20:56:37.444 INFO:teuthology.orchestra.run.smithi098.stdout:(138/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 893 kB/s | 90 kB 00:00 2024-09-10T20:56:37.469 INFO:teuthology.orchestra.run.smithi098.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-09-10T20:56:37.494 INFO:teuthology.orchestra.run.smithi098.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-10T20:56:37.519 INFO:teuthology.orchestra.run.smithi098.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-10T20:56:37.545 INFO:teuthology.orchestra.run.smithi098.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-10T20:56:37.570 INFO:teuthology.orchestra.run.smithi098.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-09-10T20:56:37.595 INFO:teuthology.orchestra.run.smithi098.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-10T20:56:37.621 INFO:teuthology.orchestra.run.smithi098.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 801 kB/s | 20 kB 00:00 2024-09-10T20:56:37.646 INFO:teuthology.orchestra.run.smithi098.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-10T20:56:37.671 INFO:teuthology.orchestra.run.smithi098.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-10T20:56:37.697 INFO:teuthology.orchestra.run.smithi098.stdout:(148/157): re2-20211101-20.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-10T20:56:37.855 INFO:teuthology.orchestra.run.smithi098.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-09-10T20:56:37.956 INFO:teuthology.orchestra.run.smithi098.stdout:(150/157): librados2-19.3.0-4854.g8f219c47.el9. 12 MB/s | 3.5 MB 00:00 2024-09-10T20:56:38.098 INFO:teuthology.orchestra.run.smithi098.stdout:(151/157): NetworkManager-1.49.90-1.el9.x86_64. 9.6 MB/s | 2.3 MB 00:00 2024-09-10T20:56:38.139 INFO:teuthology.orchestra.run.smithi098.stdout:(152/157): NetworkManager-libnm-1.49.90-1.el9.x 9.8 MB/s | 1.8 MB 00:00 2024-09-10T20:56:38.190 INFO:teuthology.orchestra.run.smithi098.stdout:(153/157): NetworkManager-team-1.49.90-1.el9.x8 444 kB/s | 41 kB 00:00 2024-09-10T20:56:38.248 INFO:teuthology.orchestra.run.smithi098.stdout:(154/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 5.9 MB/s | 3.2 MB 00:00 2024-09-10T20:56:38.274 INFO:teuthology.orchestra.run.smithi098.stdout:(155/157): NetworkManager-tui-1.49.90-1.el9.x86 1.8 MB/s | 249 kB 00:00 2024-09-10T20:56:38.357 INFO:teuthology.orchestra.run.smithi098.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 8.1 MB/s | 1.4 MB 00:00 2024-09-10T20:56:38.441 INFO:teuthology.orchestra.run.smithi098.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-09-10T20:56:38.449 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:56:38.449 INFO:teuthology.orchestra.run.smithi098.stdout:Total 11 MB/s | 240 MB 00:21 2024-09-10T20:56:39.695 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-10T20:56:39.847 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-10T20:56:39.847 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-10T20:56:42.156 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-10T20:56:42.156 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-10T20:56:43.585 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-09-10T20:56:43.685 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T20:56:43.735 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T20:56:43.798 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T20:56:43.835 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T20:56:43.861 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T20:56:43.935 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T20:56:44.262 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T20:56:44.362 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:56:44.485 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:56:44.522 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:56:44.618 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:56:44.652 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T20:56:44.682 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T20:56:44.864 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T20:56:44.915 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T20:56:44.943 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:56:45.064 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:56:45.086 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T20:56:45.133 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T20:56:45.162 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T20:56:45.188 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T20:56:45.376 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T20:56:45.544 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T20:56:45.654 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T20:56:45.794 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:56:45.858 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:56:46.086 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:56:46.201 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T20:56:46.229 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T20:56:46.271 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T20:56:46.344 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T20:56:46.389 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T20:56:46.421 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T20:56:46.464 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T20:56:46.497 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T20:56:46.525 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T20:56:46.593 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T20:56:46.647 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T20:56:46.687 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T20:56:46.738 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T20:56:46.814 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T20:56:46.849 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T20:56:46.893 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T20:56:46.921 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T20:56:46.991 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T20:56:47.036 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T20:56:47.110 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T20:56:47.244 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T20:56:47.305 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T20:56:47.833 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T20:56:47.917 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T20:56:47.946 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T20:56:47.998 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T20:56:48.044 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T20:56:48.086 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T20:56:48.254 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T20:56:48.322 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T20:56:48.414 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T20:56:48.478 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T20:56:48.567 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T20:56:48.571 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-10T20:56:48.607 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T20:56:48.643 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T20:56:48.653 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T20:56:48.690 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T20:56:48.726 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T20:56:48.769 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T20:56:48.811 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T20:56:48.829 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T20:56:48.947 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T20:56:48.988 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T20:56:48.991 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T20:56:49.030 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T20:56:49.055 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T20:56:49.105 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T20:56:49.130 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T20:56:49.335 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T20:56:49.372 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T20:56:49.429 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T20:56:49.441 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T20:56:49.462 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T20:56:49.487 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T20:56:49.556 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:56:49.592 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T20:56:49.669 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:56:49.699 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:56:49.719 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T20:56:49.753 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T20:56:49.786 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T20:56:49.801 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:56:49.823 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T20:56:49.831 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T20:56:49.862 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T20:56:49.871 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T20:56:49.902 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T20:56:49.932 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T20:56:49.966 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T20:56:50.000 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T20:56:50.035 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T20:56:50.041 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T20:56:50.076 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T20:56:50.088 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T20:56:50.116 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:56:50.162 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T20:56:50.206 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T20:56:50.221 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:56:50.246 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T20:56:50.250 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T20:56:50.313 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T20:56:50.335 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T20:56:50.349 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T20:56:50.361 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T20:56:50.405 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T20:56:50.563 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T20:56:50.724 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T20:56:50.891 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T20:56:51.001 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:56:51.077 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:56:51.121 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T20:56:51.169 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T20:56:51.206 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T20:56:51.302 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:56:51.305 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T20:56:51.361 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T20:56:51.406 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T20:56:51.415 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T20:56:51.448 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T20:56:51.481 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T20:56:51.501 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T20:56:51.542 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T20:56:51.545 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T20:56:51.616 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T20:56:51.657 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T20:56:51.699 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T20:56:51.725 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T20:56:51.733 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T20:56:51.757 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T20:56:51.761 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T20:56:51.829 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T20:56:51.845 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T20:56:51.883 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T20:56:51.915 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T20:56:51.958 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T20:56:52.043 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T20:56:52.085 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T20:56:52.128 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T20:56:52.157 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T20:56:52.226 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T20:56:52.271 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T20:56:52.346 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T20:56:52.478 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T20:56:52.508 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T20:56:52.532 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T20:56:52.629 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T20:56:53.061 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T20:56:53.137 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T20:56:53.166 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T20:56:53.218 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T20:56:53.256 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T20:56:53.297 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T20:56:53.455 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T20:56:53.509 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T20:56:53.600 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T20:56:53.673 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T20:56:53.746 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T20:56:53.778 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T20:56:53.814 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T20:56:53.860 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T20:56:53.872 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T20:56:53.896 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T20:56:53.940 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T20:56:53.947 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T20:56:53.975 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T20:56:53.989 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T20:56:54.008 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T20:56:54.134 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T20:56:54.329 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T20:56:54.351 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T20:56:54.371 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T20:56:54.444 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T20:56:54.476 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T20:56:54.476 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T20:56:54.512 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T20:56:54.578 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T20:56:54.609 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T20:56:54.657 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T20:56:54.699 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T20:56:54.732 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T20:56:54.838 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T20:56:54.967 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T20:56:55.006 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T20:56:55.007 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T20:56:55.035 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T20:56:55.040 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T20:56:55.076 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T20:56:55.111 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T20:56:55.116 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T20:56:55.142 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T20:56:55.148 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T20:56:55.178 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T20:56:55.212 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T20:56:55.254 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T20:56:55.304 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T20:56:55.338 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T20:56:55.421 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T20:56:55.472 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T20:56:55.504 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T20:56:55.606 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T20:56:55.656 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T20:56:56.379 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T20:56:56.436 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T20:56:56.470 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T20:56:56.569 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T20:56:56.625 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T20:56:56.670 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T20:56:56.757 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T20:56:56.797 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T20:56:56.981 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T20:56:57.012 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T20:56:57.090 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T20:56:57.488 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:56:57.522 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:56:57.740 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T20:56:57.865 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T20:56:58.083 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:56:58.125 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T20:56:58.216 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T20:56:58.216 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T20:56:58.216 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T20:56:58.216 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:56:58.252 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T20:56:59.129 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T20:56:59.214 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T20:56:59.241 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T20:56:59.276 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T20:56:59.596 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T20:56:59.638 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T20:56:59.738 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T20:56:59.792 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T20:56:59.837 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T20:57:00.331 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T20:57:00.366 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T20:57:00.437 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T20:57:00.476 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T20:57:02.819 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:57:02.856 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:57:03.406 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:57:03.451 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T20:57:03.528 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T20:57:03.528 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T20:57:03.529 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T20:57:03.529 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:03.577 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T20:57:10.161 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T20:57:10.162 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2024-09-10T20:57:10.162 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2024-09-10T20:57:10.162 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2024-09-10T20:57:10.162 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2024-09-10T20:57:10.162 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2024-09-10T20:57:10.162 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /root 2024-09-10T20:57:10.162 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2024-09-10T20:57:10.162 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:10.341 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T20:57:10.358 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T20:57:10.358 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:10.358 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T20:57:10.358 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:11.878 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T20:57:11.905 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T20:57:11.905 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:11.905 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T20:57:11.905 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T20:57:11.905 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T20:57:11.905 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:12.129 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T20:57:12.152 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T20:57:12.152 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:12.152 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T20:57:12.152 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T20:57:12.153 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T20:57:12.153 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:12.613 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T20:57:12.636 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T20:57:12.636 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:12.636 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T20:57:12.636 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T20:57:12.637 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T20:57:12.637 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:12.767 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T20:57:12.816 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T20:57:12.880 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T20:57:13.074 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T20:57:13.122 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T20:57:13.191 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T20:57:13.279 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T20:57:13.419 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T20:57:13.464 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T20:57:13.496 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T20:57:13.523 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T20:57:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T20:57:13.980 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T20:57:14.012 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T20:57:14.078 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T20:57:14.103 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T20:57:15.186 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T20:57:15.186 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /sys 2024-09-10T20:57:15.186 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /proc 2024-09-10T20:57:15.186 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /mnt 2024-09-10T20:57:15.186 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /var/tmp 2024-09-10T20:57:15.186 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /home 2024-09-10T20:57:15.186 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /root 2024-09-10T20:57:15.187 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /tmp 2024-09-10T20:57:15.187 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:15.357 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T20:57:15.372 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T20:57:15.372 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:15.372 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T20:57:15.372 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:16.901 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T20:57:16.928 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T20:57:16.928 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:16.929 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T20:57:16.929 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T20:57:16.929 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T20:57:16.929 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:17.153 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T20:57:17.177 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T20:57:17.177 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:17.177 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T20:57:17.177 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T20:57:17.177 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T20:57:17.177 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:17.297 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T20:57:17.360 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T20:57:17.637 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T20:57:17.662 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T20:57:17.662 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:17.662 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T20:57:17.662 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T20:57:17.662 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T20:57:17.663 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:17.768 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T20:57:17.816 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T20:57:17.879 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T20:57:18.065 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T20:57:18.114 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T20:57:18.183 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T20:57:18.271 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T20:57:18.333 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T20:57:18.360 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T20:57:18.417 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T20:57:18.456 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T20:57:18.473 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T20:57:18.488 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T20:57:18.515 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T20:57:18.640 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T20:57:18.662 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T20:57:18.687 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T20:57:18.687 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:18.687 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T20:57:18.687 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T20:57:18.687 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T20:57:18.687 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:18.737 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T20:57:18.749 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T20:57:18.848 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T20:57:18.898 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T20:57:18.938 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T20:57:18.964 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T20:57:18.996 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T20:57:19.062 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T20:57:19.087 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T20:57:19.854 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T20:57:19.881 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T20:57:19.906 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T20:57:19.906 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:19.906 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T20:57:19.906 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T20:57:19.906 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T20:57:19.906 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:19.949 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T20:57:19.972 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T20:57:19.972 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:19.972 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T20:57:19.972 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:20.016 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T20:57:20.039 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T20:57:20.039 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:20.039 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T20:57:20.039 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T20:57:20.039 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T20:57:20.040 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:20.350 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T20:57:20.374 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T20:57:20.374 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:20.374 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T20:57:20.374 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T20:57:20.374 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T20:57:20.375 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:22.264 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T20:57:22.319 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T20:57:23.287 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T20:57:23.319 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T20:57:23.432 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T20:57:23.607 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T20:57:23.641 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T20:57:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T20:57:23.663 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:23.663 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T20:57:23.663 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T20:57:23.663 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T20:57:23.663 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:23.713 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T20:57:23.725 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T20:57:23.832 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T20:57:23.887 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T20:57:24.843 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T20:57:24.868 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T20:57:24.890 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T20:57:24.890 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:24.890 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T20:57:24.890 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T20:57:24.890 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T20:57:24.890 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:24.942 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T20:57:24.965 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T20:57:24.965 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:24.966 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T20:57:24.966 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:25.009 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T20:57:25.033 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T20:57:25.033 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:25.033 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T20:57:25.033 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T20:57:25.033 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T20:57:25.033 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:25.122 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T20:57:25.162 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T20:57:25.203 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T20:57:25.227 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T20:57:25.254 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T20:57:25.355 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T20:57:25.379 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T20:57:25.379 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:57:25.379 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T20:57:25.380 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T20:57:25.380 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T20:57:25.380 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:25.439 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T20:57:25.492 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T20:57:25.523 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T20:57:25.552 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T20:57:25.607 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T20:57:26.163 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:57:26.205 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:57:26.225 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:57:27.044 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T20:57:27.126 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T20:57:27.177 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T20:57:27.266 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T20:57:27.305 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T20:57:27.330 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T20:57:27.356 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T20:57:27.356 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T20:57:27.448 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T20:57:27.448 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T20:57:27.532 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T20:57:27.557 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T20:57:27.563 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:57:27.563 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:57:27.751 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:57:27.751 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T20:57:27.835 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T20:57:27.908 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:57:30.136 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T20:57:30.180 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T20:57:30.222 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T20:57:30.254 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T20:57:30.281 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T20:57:30.469 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T20:57:30.511 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T20:57:30.550 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T20:57:30.588 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T20:57:30.643 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T20:57:31.242 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:57:31.282 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:57:31.303 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:57:32.129 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T20:57:32.221 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T20:57:32.268 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T20:57:32.355 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T20:57:32.399 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T20:57:32.424 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T20:57:32.451 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T20:57:32.451 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T20:57:32.543 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T20:57:32.543 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T20:57:32.627 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T20:57:32.652 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T20:57:32.658 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:57:32.658 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:57:32.838 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:57:32.838 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T20:57:32.913 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T20:57:32.978 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:57:34.247 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:57:34.247 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T20:57:34.247 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T20:57:34.248 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T20:57:34.248 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T20:57:34.248 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T20:57:34.248 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T20:57:34.248 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T20:57:34.248 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:57:34.248 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:57:34.248 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T20:57:34.248 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T20:57:34.248 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T20:57:34.248 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T20:57:34.248 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:57:34.249 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T20:57:34.249 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T20:57:34.249 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T20:57:34.249 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T20:57:34.249 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T20:57:34.249 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T20:57:34.249 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T20:57:34.249 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T20:57:34.249 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T20:57:34.249 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T20:57:34.249 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T20:57:34.249 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T20:57:34.249 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T20:57:34.250 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T20:57:34.250 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T20:57:34.250 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T20:57:34.262 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T20:57:34.262 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T20:57:34.262 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T20:57:34.262 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T20:57:34.262 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T20:57:34.262 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T20:57:34.262 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T20:57:34.262 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T20:57:34.262 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T20:57:34.263 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T20:57:34.263 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T20:57:34.263 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T20:57:34.263 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T20:57:34.263 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T20:57:34.263 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T20:57:34.263 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T20:57:34.263 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T20:57:34.263 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T20:57:34.263 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T20:57:34.263 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T20:57:34.263 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T20:57:34.263 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T20:57:34.264 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T20:57:34.264 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T20:57:34.264 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T20:57:34.264 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T20:57:34.264 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T20:57:34.264 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T20:57:34.264 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T20:57:34.264 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T20:57:34.264 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T20:57:34.264 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T20:57:34.264 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T20:57:34.264 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T20:57:34.265 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T20:57:34.265 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T20:57:34.265 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T20:57:34.265 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T20:57:34.265 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T20:57:34.265 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T20:57:34.265 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T20:57:34.265 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T20:57:34.265 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T20:57:34.265 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T20:57:34.265 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T20:57:34.265 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T20:57:34.266 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T20:57:34.266 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T20:57:34.266 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T20:57:34.266 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T20:57:34.266 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T20:57:34.266 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T20:57:34.266 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T20:57:34.266 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T20:57:34.266 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T20:57:34.266 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T20:57:34.266 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T20:57:34.266 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T20:57:34.266 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T20:57:34.267 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T20:57:34.267 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T20:57:34.267 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T20:57:34.267 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T20:57:34.267 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T20:57:34.267 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T20:57:34.267 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T20:57:34.267 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T20:57:34.267 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T20:57:34.267 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T20:57:34.267 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T20:57:34.267 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T20:57:34.267 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T20:57:34.267 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T20:57:34.268 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T20:57:34.268 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T20:57:34.268 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T20:57:34.268 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T20:57:34.268 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T20:57:34.268 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T20:57:34.268 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T20:57:34.268 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T20:57:34.268 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T20:57:34.268 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T20:57:34.268 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T20:57:34.268 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T20:57:34.268 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T20:57:34.269 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T20:57:34.269 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T20:57:34.269 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T20:57:34.269 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T20:57:34.269 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T20:57:34.269 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T20:57:34.269 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T20:57:34.269 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T20:57:34.269 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T20:57:34.269 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T20:57:34.269 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T20:57:34.269 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T20:57:34.269 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T20:57:34.270 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T20:57:34.270 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T20:57:34.270 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T20:57:34.270 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T20:57:34.270 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T20:57:34.270 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T20:57:34.270 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T20:57:34.270 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T20:57:34.270 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T20:57:34.270 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T20:57:34.270 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T20:57:34.270 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T20:57:34.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T20:57:34.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T20:57:34.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T20:57:34.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T20:57:34.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T20:57:34.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T20:57:34.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T20:57:34.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T20:57:34.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T20:57:34.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T20:57:34.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T20:57:34.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T20:57:34.271 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T20:57:34.272 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T20:57:34.272 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T20:57:34.272 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T20:57:34.272 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T20:57:34.272 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T20:57:34.272 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T20:57:34.272 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T20:57:34.272 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T20:57:34.272 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T20:57:34.272 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T20:57:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:57:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:36.728 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2024-09-10T20:57:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T20:57:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T20:57:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T20:57:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T20:57:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T20:57:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T20:57:36.729 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-09-10T20:57:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T20:57:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T20:57:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T20:57:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T20:57:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T20:57:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T20:57:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T20:57:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T20:57:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T20:57:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T20:57:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T20:57:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T20:57:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T20:57:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T20:57:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T20:57:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T20:57:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T20:57:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T20:57:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T20:57:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T20:57:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T20:57:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T20:57:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T20:57:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T20:57:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T20:57:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T20:57:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T20:57:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T20:57:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T20:57:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T20:57:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T20:57:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T20:57:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T20:57:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T20:57:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T20:57:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T20:57:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T20:57:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T20:57:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T20:57:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T20:57:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T20:57:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T20:57:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T20:57:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T20:57:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T20:57:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T20:57:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T20:57:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T20:57:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T20:57:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T20:57:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T20:57:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T20:57:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T20:57:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T20:57:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T20:57:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T20:57:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T20:57:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T20:57:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T20:57:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T20:57:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T20:57:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T20:57:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T20:57:36.740 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T20:57:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T20:57:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T20:57:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T20:57:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T20:57:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T20:57:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T20:57:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T20:57:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T20:57:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T20:57:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T20:57:36.741 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T20:57:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T20:57:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T20:57:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T20:57:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T20:57:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T20:57:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T20:57:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T20:57:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T20:57:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T20:57:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T20:57:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.742 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T20:57:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T20:57:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T20:57:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T20:57:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T20:57:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T20:57:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T20:57:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T20:57:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T20:57:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T20:57:36.743 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T20:57:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T20:57:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T20:57:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T20:57:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T20:57:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T20:57:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T20:57:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T20:57:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:36.744 INFO:teuthology.orchestra.run.smithi190.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T20:57:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T20:57:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T20:57:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T20:57:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T20:57:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T20:57:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T20:57:36.745 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:36.746 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-09-10T20:57:37.182 DEBUG:teuthology.parallel:result is None 2024-09-10T20:57:39.397 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:57:39.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T20:57:39.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T20:57:39.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T20:57:39.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T20:57:39.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T20:57:39.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T20:57:39.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T20:57:39.397 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:57:39.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:57:39.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T20:57:39.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T20:57:39.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T20:57:39.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T20:57:39.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:57:39.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T20:57:39.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T20:57:39.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T20:57:39.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T20:57:39.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T20:57:39.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T20:57:39.398 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T20:57:39.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T20:57:39.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T20:57:39.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T20:57:39.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T20:57:39.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T20:57:39.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T20:57:39.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T20:57:39.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T20:57:39.399 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T20:57:39.413 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T20:57:39.414 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T20:57:39.414 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T20:57:39.414 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T20:57:39.414 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T20:57:39.414 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T20:57:39.414 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T20:57:39.414 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T20:57:39.414 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T20:57:39.414 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T20:57:39.414 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T20:57:39.414 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T20:57:39.414 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T20:57:39.414 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T20:57:39.415 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T20:57:39.415 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T20:57:39.415 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T20:57:39.415 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T20:57:39.415 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T20:57:39.415 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T20:57:39.415 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T20:57:39.415 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T20:57:39.415 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T20:57:39.415 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T20:57:39.415 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T20:57:39.415 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T20:57:39.416 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T20:57:39.416 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T20:57:39.416 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T20:57:39.416 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T20:57:39.416 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T20:57:39.416 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T20:57:39.416 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T20:57:39.416 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T20:57:39.416 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T20:57:39.416 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T20:57:39.416 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T20:57:39.417 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T20:57:39.417 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T20:57:39.417 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T20:57:39.417 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T20:57:39.417 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T20:57:39.417 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T20:57:39.417 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T20:57:39.417 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T20:57:39.417 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T20:57:39.417 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T20:57:39.418 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T20:57:39.418 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T20:57:39.418 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T20:57:39.418 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T20:57:39.418 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T20:57:39.418 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T20:57:39.418 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T20:57:39.418 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T20:57:39.418 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T20:57:39.418 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T20:57:39.418 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T20:57:39.418 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T20:57:39.418 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T20:57:39.419 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T20:57:39.419 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T20:57:39.419 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T20:57:39.419 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T20:57:39.419 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T20:57:39.419 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T20:57:39.419 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T20:57:39.419 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T20:57:39.419 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T20:57:39.419 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T20:57:39.419 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T20:57:39.419 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T20:57:39.419 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T20:57:39.419 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T20:57:39.420 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T20:57:39.420 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T20:57:39.420 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T20:57:39.420 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T20:57:39.420 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T20:57:39.420 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T20:57:39.420 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T20:57:39.420 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T20:57:39.420 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T20:57:39.420 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T20:57:39.420 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T20:57:39.420 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T20:57:39.420 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T20:57:39.420 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T20:57:39.421 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T20:57:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T20:57:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T20:57:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T20:57:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T20:57:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T20:57:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T20:57:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T20:57:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T20:57:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T20:57:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T20:57:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T20:57:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T20:57:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T20:57:39.422 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T20:57:39.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T20:57:39.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T20:57:39.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T20:57:39.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T20:57:39.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T20:57:39.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T20:57:39.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T20:57:39.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T20:57:39.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T20:57:39.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T20:57:39.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T20:57:39.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T20:57:39.423 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T20:57:39.424 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T20:57:39.424 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T20:57:39.424 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T20:57:39.424 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T20:57:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:57:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:41.987 INFO:teuthology.orchestra.run.smithi098.stdout:Upgraded: 2024-09-10T20:57:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T20:57:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T20:57:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T20:57:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T20:57:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T20:57:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T20:57:41.988 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-10T20:57:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T20:57:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T20:57:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T20:57:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T20:57:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.988 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.989 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:57:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T20:57:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T20:57:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T20:57:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T20:57:41.990 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T20:57:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T20:57:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T20:57:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T20:57:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T20:57:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T20:57:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T20:57:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T20:57:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T20:57:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T20:57:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T20:57:41.991 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T20:57:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T20:57:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T20:57:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T20:57:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T20:57:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T20:57:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T20:57:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T20:57:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T20:57:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T20:57:41.992 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T20:57:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T20:57:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T20:57:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T20:57:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T20:57:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T20:57:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T20:57:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T20:57:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T20:57:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T20:57:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T20:57:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T20:57:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T20:57:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T20:57:41.993 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T20:57:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T20:57:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T20:57:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T20:57:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T20:57:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T20:57:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T20:57:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T20:57:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T20:57:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T20:57:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T20:57:41.994 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T20:57:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T20:57:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T20:57:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T20:57:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T20:57:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T20:57:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T20:57:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T20:57:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T20:57:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T20:57:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T20:57:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T20:57:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T20:57:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T20:57:41.995 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T20:57:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T20:57:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T20:57:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T20:57:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T20:57:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T20:57:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T20:57:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T20:57:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T20:57:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T20:57:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T20:57:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T20:57:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T20:57:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T20:57:41.996 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T20:57:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T20:57:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T20:57:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T20:57:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T20:57:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T20:57:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T20:57:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T20:57:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T20:57:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T20:57:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T20:57:41.997 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T20:57:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T20:57:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T20:57:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T20:57:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T20:57:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T20:57:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T20:57:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T20:57:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T20:57:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T20:57:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:57:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T20:57:41.998 INFO:teuthology.orchestra.run.smithi098.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T20:57:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T20:57:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T20:57:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T20:57:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T20:57:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T20:57:41.999 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:41.999 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-10T20:57:42.446 DEBUG:teuthology.parallel:result is None 2024-09-10T20:57:42.446 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:57:42.598 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T20:57:42.620 INFO:teuthology.orchestra.run.smithi098.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T20:57:42.621 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T20:57:42.621 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T20:57:42.623 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:57:42.771 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T20:57:42.798 INFO:teuthology.orchestra.run.smithi190.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T20:57:42.798 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T20:57:42.798 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T20:57:42.800 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T20:57:42.800 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:57:42.800 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T20:57:42.828 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:57:42.828 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T20:57:42.868 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T20:57:42.868 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:57:42.868 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T20:57:42.895 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T20:57:42.962 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:57:42.962 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T20:57:42.990 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T20:57:43.058 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T20:57:43.058 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:57:43.059 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T20:57:43.085 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T20:57:43.150 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:57:43.150 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T20:57:43.178 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T20:57:43.244 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T20:57:43.244 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:57:43.244 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T20:57:43.269 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T20:57:43.335 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:57:43.335 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T20:57:43.361 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T20:57:43.426 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T20:57:43.516 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T20:57:43.517 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 777 /var/log/ceph 2024-09-10T20:57:43.519 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /var/log/ceph 2024-09-10T20:57:43.544 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T20:57:43.544 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T20:57:43.584 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T20:57:43.609 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T20:57:43.609 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T20:57:43.649 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T20:57:43.680 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-10T20:57:43.680 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-10T20:57:43.680 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-10T20:57:43.680 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:57:43.680 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T20:57:43.727 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T20:57:43.841 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-10T20:57:43.866 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:57:43.866 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T20:57:43.881 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T20:57:44.001 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-10T20:57:44.038 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T20:57:44.038 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Replacing daemon mds.a'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T20:57:44.039 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899652', 'branch': 'wip-hemanth-testing-2024-09-10-0723', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/cap-flush}', 'email': 'hyelloji@redhat.com', 'first_in_suite': False, 'job_id': '7899652', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-hemanth-testing-2024-09-10-0723'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Replacing daemon mds.a'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-hemanth-testing-2024-09-10-0723', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'owner': 'scheduled_hyelloji@teuthology', 'priority': 100, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6190, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'sleep_before_teardown': 0, 'subset': '479/1024', 'suite': 'fs', 'suite_branch': 'wip-hemanth-testing-2024-09-10-0723', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi098.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBERnTf0dpCtnGZSeXc+jviUtasaLkYOu1WhRA8LTXF1C52RVaVi8QT9PLudcJarM74AlqUqrrTmkxgS+h8G6/lg=', 'smithi190.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFQiF0McfLgNBxTrH+/J5mvaNf8W/eSVcmYR9y3gnU3TbDAatBiMJUglgn7cPBh9lCfJ378lRhED+Qbz5Fv2Mc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Replacing daemon mds.a'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_cap_flush']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_16:59:47', 'tube': 'smithi', 'user': 'hyelloji', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T20:57:44.039 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T20:57:44.041 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T20:57:44.089 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T20:57:44.097 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T20:57:44.154 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:57:44.154 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T20:57:44.174 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T20:57:44.175 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_1 2024-09-10T20:57:44.230 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T20:57:44.230 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:57:44.230 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-10T20:57:44.230 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:57:44.230 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:57:44.230 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-10 20:57:33.353631817 +0000 2024-09-10T20:57:44.230 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-10 20:55:35.840710441 +0000 2024-09-10T20:57:44.230 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-10 20:55:35.840710441 +0000 2024-09-10T20:57:44.230 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-10 20:55:35.840710441 +0000 2024-09-10T20:57:44.231 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T20:57:44.292 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-10T20:57:44.292 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-10T20:57:44.292 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000200669 s, 2.6 MB/s 2024-09-10T20:57:44.293 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T20:57:44.350 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_2 2024-09-10T20:57:44.406 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T20:57:44.406 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:57:44.406 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-10T20:57:44.406 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:57:44.406 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:57:44.406 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-10 20:57:33.353631817 +0000 2024-09-10T20:57:44.406 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-10 20:55:36.168706488 +0000 2024-09-10T20:57:44.406 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-10 20:55:36.168706488 +0000 2024-09-10T20:57:44.406 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-10 20:55:36.168706488 +0000 2024-09-10T20:57:44.406 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T20:57:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-10T20:57:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-10T20:57:44.469 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000175956 s, 2.9 MB/s 2024-09-10T20:57:44.470 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T20:57:44.527 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_3 2024-09-10T20:57:44.582 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T20:57:44.583 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:57:44.583 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-10T20:57:44.583 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:57:44.583 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:57:44.583 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-10 20:57:33.353631817 +0000 2024-09-10T20:57:44.583 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-10 20:55:36.493702572 +0000 2024-09-10T20:57:44.583 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-10 20:55:36.493702572 +0000 2024-09-10T20:57:44.583 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-10 20:55:36.493702572 +0000 2024-09-10T20:57:44.583 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T20:57:44.645 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-10T20:57:44.646 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-10T20:57:44.646 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000171637 s, 3.0 MB/s 2024-09-10T20:57:44.647 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T20:57:44.702 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_4 2024-09-10T20:57:44.758 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T20:57:44.758 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:57:44.758 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-10T20:57:44.758 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:57:44.758 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:57:44.758 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-10 20:57:33.354631804 +0000 2024-09-10T20:57:44.758 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-10 20:55:36.828698534 +0000 2024-09-10T20:57:44.759 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-10 20:55:36.828698534 +0000 2024-09-10T20:57:44.759 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-10 20:55:36.828698534 +0000 2024-09-10T20:57:44.759 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T20:57:44.822 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-10T20:57:44.822 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-10T20:57:44.822 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000192967 s, 2.7 MB/s 2024-09-10T20:57:44.823 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T20:57:44.881 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T20:57:44.881 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:57:44.881 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T20:57:44.900 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T20:57:44.900 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-09-10T20:57:44.958 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T20:57:44.958 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:57:44.958 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-10T20:57:44.958 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:57:44.958 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:57:44.958 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-10 20:57:28.273180414 +0000 2024-09-10T20:57:44.958 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-10 20:55:36.686577978 +0000 2024-09-10T20:57:44.958 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-10 20:55:36.686577978 +0000 2024-09-10T20:57:44.958 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-10 20:55:36.686577978 +0000 2024-09-10T20:57:44.959 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T20:57:45.024 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-10T20:57:45.024 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-10T20:57:45.024 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000252731 s, 2.0 MB/s 2024-09-10T20:57:45.026 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T20:57:45.084 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-09-10T20:57:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T20:57:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:57:45.142 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-10T20:57:45.142 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:57:45.142 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:57:45.142 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-10 20:57:28.274180386 +0000 2024-09-10T20:57:45.142 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-10 20:55:37.018568520 +0000 2024-09-10T20:57:45.142 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-10 20:55:37.018568520 +0000 2024-09-10T20:57:45.142 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-10 20:55:37.018568520 +0000 2024-09-10T20:57:45.142 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T20:57:45.206 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-10T20:57:45.207 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-10T20:57:45.207 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000196401 s, 2.6 MB/s 2024-09-10T20:57:45.208 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T20:57:45.264 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-09-10T20:57:45.320 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T20:57:45.321 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:57:45.321 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-10T20:57:45.321 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:57:45.321 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:57:45.321 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-10 20:57:28.274180386 +0000 2024-09-10T20:57:45.321 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-10 20:55:37.353558977 +0000 2024-09-10T20:57:45.321 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-10 20:55:37.353558977 +0000 2024-09-10T20:57:45.321 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-10 20:55:37.353558977 +0000 2024-09-10T20:57:45.321 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T20:57:45.385 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-10T20:57:45.385 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-10T20:57:45.385 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000219381 s, 2.3 MB/s 2024-09-10T20:57:45.386 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T20:57:45.443 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-09-10T20:57:45.498 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T20:57:45.498 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:57:45.498 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-10T20:57:45.498 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:57:45.498 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:57:45.498 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-09-10 20:57:28.274180386 +0000 2024-09-10T20:57:45.498 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-09-10 20:55:37.689549405 +0000 2024-09-10T20:57:45.499 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-09-10 20:55:37.689549405 +0000 2024-09-10T20:57:45.499 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-09-10 20:55:37.689549405 +0000 2024-09-10T20:57:45.499 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T20:57:45.561 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-09-10T20:57:45.562 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-09-10T20:57:45.562 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000171774 s, 3.0 MB/s 2024-09-10T20:57:45.563 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T20:57:45.619 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T20:57:45.619 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi098.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T20:57:45.619 INFO:tasks.ceph:Generating config... 2024-09-10T20:57:45.621 INFO:tasks.ceph:[client] pid file = 2024-09-10T20:57:45.621 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-10T20:57:45.621 INFO:tasks.ceph:[mgr] client mount timeout = 30 2024-09-10T20:57:45.621 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-10T20:57:45.621 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T20:57:45.621 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T20:57:45.621 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T20:57:45.621 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T20:57:45.621 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T20:57:45.621 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-10T20:57:45.621 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T20:57:45.621 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T20:57:45.621 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-10T20:57:45.622 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-10T20:57:45.622 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T20:57:45.622 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T20:57:45.622 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-10T20:57:45.622 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T20:57:45.622 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T20:57:45.622 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T20:57:45.622 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T20:57:45.622 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T20:57:45.622 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T20:57:45.622 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T20:57:45.687 INFO:teuthology.orchestra.run.smithi098.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T20:57:45.691 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T20:57:45.735 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T20:57:45.760 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.98'), ('mon.b', '172.21.15.190'), ('mon.c', '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]')] 2024-09-10T20:57:45.760 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.98,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'client mount timeout': 30, 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-10T20:57:45.761 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:57:45.761 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T20:57:45.815 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.98 --add b 172.21.15.190 --addv c '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:57:45.901 INFO:teuthology.orchestra.run.smithi098.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:57:45.901 INFO:teuthology.orchestra.run.smithi098.stdout:monmaptool: generated fsid 41041377-a016-49fd-be1f-323d0b834557 2024-09-10T20:57:45.901 INFO:teuthology.orchestra.run.smithi098.stdout:setting min_mon_release = quincy 2024-09-10T20:57:45.901 INFO:teuthology.orchestra.run.smithi098.stdout:epoch 0 2024-09-10T20:57:45.902 INFO:teuthology.orchestra.run.smithi098.stdout:fsid 41041377-a016-49fd-be1f-323d0b834557 2024-09-10T20:57:45.902 INFO:teuthology.orchestra.run.smithi098.stdout:last_changed 2024-09-10T20:57:45.898200+0000 2024-09-10T20:57:45.902 INFO:teuthology.orchestra.run.smithi098.stdout:created 2024-09-10T20:57:45.898200+0000 2024-09-10T20:57:45.902 INFO:teuthology.orchestra.run.smithi098.stdout:min_mon_release 17 (quincy) 2024-09-10T20:57:45.902 INFO:teuthology.orchestra.run.smithi098.stdout:election_strategy: 1 2024-09-10T20:57:45.902 INFO:teuthology.orchestra.run.smithi098.stdout:0: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.a 2024-09-10T20:57:45.902 INFO:teuthology.orchestra.run.smithi098.stdout:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2024-09-10T20:57:45.902 INFO:teuthology.orchestra.run.smithi098.stdout:2: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.c 2024-09-10T20:57:45.902 INFO:teuthology.orchestra.run.smithi098.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T20:57:45.903 DEBUG:teuthology.orchestra.run.smithi098:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T20:57:45.918 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 41041377-a016-49fd-be1f-323d0b834557... 2024-09-10T20:57:45.919 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T20:57:45.960 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T20:57:46.000 INFO:teuthology.orchestra.run.smithi098.stdout:[global] 2024-09-10T20:57:46.000 INFO:teuthology.orchestra.run.smithi098.stdout: chdir = "" 2024-09-10T20:57:46.000 INFO:teuthology.orchestra.run.smithi098.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T20:57:46.000 INFO:teuthology.orchestra.run.smithi098.stdout: auth supported = cephx 2024-09-10T20:57:46.000 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.000 INFO:teuthology.orchestra.run.smithi098.stdout: filestore xattr use omap = true 2024-09-10T20:57:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: mon clock drift allowed = 1.000 2024-09-10T20:57:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: osd crush chooseleaf type = 0 2024-09-10T20:57:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: auth debug = true 2024-09-10T20:57:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on old message = true 2024-09-10T20:57:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on bug = true 2024-09-10T20:57:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.001 INFO:teuthology.orchestra.run.smithi098.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T20:57:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: mon pg warn max object skew = 0 2024-09-10T20:57:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T20:57:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T20:57:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: osd pool default size = 2 2024-09-10T20:57:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd allow primary affinity = true 2024-09-10T20:57:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd allow pg remap = true 2024-09-10T20:57:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on legacy crush tunables = false 2024-09-10T20:57:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on crush straw calc version zero = false 2024-09-10T20:57:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on no sortbitwise = false 2024-09-10T20:57:46.002 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on osd down out interval zero = false 2024-09-10T20:57:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on too few osds = false 2024-09-10T20:57:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T20:57:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T20:57:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: mon_allow_pool_size_one = true 2024-09-10T20:57:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T20:57:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: osd default data pool replay window = 5 2024-09-10T20:57:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: mon allow pool delete = true 2024-09-10T20:57:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.003 INFO:teuthology.orchestra.run.smithi098.stdout: mon cluster log file level = debug 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: debug asserts on shutdown = true 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: mon health detail to clog = false 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: mon host = "172.21.15.98,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on skipped message = False 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: fsid = 41041377-a016-49fd-be1f-323d0b834557 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout:[osd] 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: osd journal size = 100 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: osd scrub load threshold = 5.0 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: osd scrub max interval = 600 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: osd mclock profile = high_recovery_ops 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: osd recover clone overlap = true 2024-09-10T20:57:46.004 INFO:teuthology.orchestra.run.smithi098.stdout: osd recovery max chunk = 1048576 2024-09-10T20:57:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug shutdown = true 2024-09-10T20:57:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug op order = true 2024-09-10T20:57:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug verify stray on activate = true 2024-09-10T20:57:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug trim objects = true 2024-09-10T20:57:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: osd open classes on start = true 2024-09-10T20:57:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug pg log writeout = true 2024-09-10T20:57:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: osd deep scrub update digest min age = 30 2024-09-10T20:57:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: osd map max advance = 10 2024-09-10T20:57:46.005 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: journal zero on create = true 2024-09-10T20:57:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: filestore ondisk finisher threads = 3 2024-09-10T20:57:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: filestore apply finisher threads = 3 2024-09-10T20:57:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: bdev debug aio = true 2024-09-10T20:57:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug misdirected ops = true 2024-09-10T20:57:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore block size = 96636764160 2024-09-10T20:57:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore fsck on mount = True 2024-09-10T20:57:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: debug bluefs = 20 2024-09-10T20:57:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: debug bluestore = 20 2024-09-10T20:57:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-09-10T20:57:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: debug osd = 20 2024-09-10T20:57:46.006 INFO:teuthology.orchestra.run.smithi098.stdout: debug rocksdb = 10 2024-09-10T20:57:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T20:57:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd full ratio = 0.9 2024-09-10T20:57:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd nearfull ratio = 0.8 2024-09-10T20:57:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: osd failsafe full ratio = 0.95 2024-09-10T20:57:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: osd objectstore = bluestore 2024-09-10T20:57:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.007 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr] 2024-09-10T20:57:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-09-10T20:57:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: debug mgr = 20 2024-09-10T20:57:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: debug mon = 20 2024-09-10T20:57:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: debug auth = 20 2024-09-10T20:57:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min pgs per osd = 4 2024-09-10T20:57:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min bytes per osd = 10 2024-09-10T20:57:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: mgr/telemetry/nag = false 2024-09-10T20:57:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: client mount timeout = 30 2024-09-10T20:57:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: debug client = 20 2024-09-10T20:57:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.008 INFO:teuthology.orchestra.run.smithi098.stdout:[mon] 2024-09-10T20:57:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-09-10T20:57:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: debug mon = 20 2024-09-10T20:57:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: debug paxos = 20 2024-09-10T20:57:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: debug auth = 20 2024-09-10T20:57:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: mon data avail warn = 5 2024-09-10T20:57:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: mon mgr mkfs grace = 240 2024-09-10T20:57:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min pgs per osd = 4 2024-09-10T20:57:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd reporter subtree level = osd 2024-09-10T20:57:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd prime pg temp = true 2024-09-10T20:57:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min bytes per osd = 10 2024-09-10T20:57:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T20:57:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T20:57:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T20:57:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T20:57:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T20:57:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T20:57:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: # 1m isn't quite enough 2024-09-10T20:57:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: mon_down_mkfs_grace = 2m 2024-09-10T20:57:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_filestore_osds = false 2024-09-10T20:57:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: mon down mkfs grace = 300 2024-09-10T20:57:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:57:46.010 INFO:teuthology.orchestra.run.smithi098.stdout:[client] 2024-09-10T20:57:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: rgw cache enabled = true 2024-09-10T20:57:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: rgw enable ops log = true 2024-09-10T20:57:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: rgw enable usage log = true 2024-09-10T20:57:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T20:57:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T20:57:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: pid file = "" 2024-09-10T20:57:46.010 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.a] 2024-09-10T20:57:46.010 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.b] 2024-09-10T20:57:46.010 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.c] 2024-09-10T20:57:46.011 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.a] 2024-09-10T20:57:46.011 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.c] 2024-09-10T20:57:46.011 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.b] 2024-09-10T20:57:46.011 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.d] 2024-09-10T20:57:46.011 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2024-09-10T20:57:46.011 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2024-09-10T20:57:46.011 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T20:57:46.011 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2024-09-10T20:57:46.012 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.012 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2024-09-10T20:57:46.012 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.012 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2024-09-10T20:57:46.012 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.012 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2024-09-10T20:57:46.012 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2024-09-10T20:57:46.012 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.012 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2024-09-10T20:57:46.012 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2024-09-10T20:57:46.012 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.012 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T20:57:46.012 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2024-09-10T20:57:46.013 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.013 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T20:57:46.013 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T20:57:46.013 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.013 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2024-09-10T20:57:46.013 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.013 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2024-09-10T20:57:46.013 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2024-09-10T20:57:46.013 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2024-09-10T20:57:46.013 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2024-09-10T20:57:46.013 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2024-09-10T20:57:46.013 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2024-09-10T20:57:46.014 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2024-09-10T20:57:46.014 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T20:57:46.014 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T20:57:46.014 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2024-09-10T20:57:46.014 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.014 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T20:57:46.014 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.014 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2024-09-10T20:57:46.014 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.014 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2024-09-10T20:57:46.014 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.014 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2024-09-10T20:57:46.014 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2024-09-10T20:57:46.015 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2024-09-10T20:57:46.015 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.98,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2024-09-10T20:57:46.015 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on skipped message = False 2024-09-10T20:57:46.015 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = 41041377-a016-49fd-be1f-323d0b834557 2024-09-10T20:57:46.015 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.015 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2024-09-10T20:57:46.015 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2024-09-10T20:57:46.015 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.015 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2024-09-10T20:57:46.015 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2024-09-10T20:57:46.015 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2024-09-10T20:57:46.015 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.015 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2024-09-10T20:57:46.015 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2024-09-10T20:57:46.016 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.016 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2024-09-10T20:57:46.016 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2024-09-10T20:57:46.016 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2024-09-10T20:57:46.016 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug trim objects = true 2024-09-10T20:57:46.016 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.016 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2024-09-10T20:57:46.016 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2024-09-10T20:57:46.016 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.016 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2024-09-10T20:57:46.016 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.016 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2024-09-10T20:57:46.016 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.017 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2024-09-10T20:57:46.017 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.017 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2024-09-10T20:57:46.017 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2024-09-10T20:57:46.017 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.017 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2024-09-10T20:57:46.017 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2024-09-10T20:57:46.017 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2024-09-10T20:57:46.017 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2024-09-10T20:57:46.017 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2024-09-10T20:57:46.017 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2024-09-10T20:57:46.017 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-09-10T20:57:46.017 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2024-09-10T20:57:46.018 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2024-09-10T20:57:46.018 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T20:57:46.018 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2024-09-10T20:57:46.018 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2024-09-10T20:57:46.018 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2024-09-10T20:57:46.018 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2024-09-10T20:57:46.018 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.018 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2024-09-10T20:57:46.018 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-09-10T20:57:46.018 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2024-09-10T20:57:46.018 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2024-09-10T20:57:46.018 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2024-09-10T20:57:46.018 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2024-09-10T20:57:46.019 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2024-09-10T20:57:46.019 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2024-09-10T20:57:46.019 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 30 2024-09-10T20:57:46.019 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2024-09-10T20:57:46.019 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.019 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2024-09-10T20:57:46.019 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-09-10T20:57:46.019 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2024-09-10T20:57:46.019 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2024-09-10T20:57:46.019 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2024-09-10T20:57:46.019 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2024-09-10T20:57:46.019 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2024-09-10T20:57:46.019 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2024-09-10T20:57:46.020 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2024-09-10T20:57:46.020 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2024-09-10T20:57:46.020 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2024-09-10T20:57:46.020 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.020 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T20:57:46.020 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T20:57:46.020 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T20:57:46.020 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.020 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T20:57:46.020 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T20:57:46.020 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T20:57:46.020 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.020 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2024-09-10T20:57:46.021 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2024-09-10T20:57:46.021 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.021 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2024-09-10T20:57:46.021 INFO:teuthology.orchestra.run.smithi190.stdout: mon down mkfs grace = 300 2024-09-10T20:57:46.021 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-09-10T20:57:46.021 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2024-09-10T20:57:46.021 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2024-09-10T20:57:46.021 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2024-09-10T20:57:46.021 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2024-09-10T20:57:46.021 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T20:57:46.021 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T20:57:46.021 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2024-09-10T20:57:46.022 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2024-09-10T20:57:46.022 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2024-09-10T20:57:46.022 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2024-09-10T20:57:46.022 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2024-09-10T20:57:46.022 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2024-09-10T20:57:46.022 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2024-09-10T20:57:46.022 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2024-09-10T20:57:46.023 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T20:57:46.024 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T20:57:46.117 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T20:57:46.117 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:57:46.117 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T20:57:46.132 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:57:46.133 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T20:57:46.188 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T20:57:46.188 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:57:46.188 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T20:57:46.188 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T20:57:46.261 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:57:46.261 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:57:46.317 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi190.front.sepia.ceph.com 2024-09-10T20:57:46.317 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:57:46.317 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T20:57:46.318 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T20:57:46.352 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:57:46.352 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:57:46.408 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T20:57:46.408 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T20:57:46.408 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T20:57:46.459 INFO:teuthology.orchestra.run.smithi098.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T20:57:46.461 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T20:57:46.524 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T20:57:46.526 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T20:57:46.526 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T20:57:46.577 INFO:teuthology.orchestra.run.smithi098.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T20:57:46.579 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-10T20:57:46.604 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T20:57:46.694 INFO:teuthology.orchestra.run.smithi098.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T20:57:46.697 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-10T20:57:46.721 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T20:57:46.773 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T20:57:46.775 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-10T20:57:46.800 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T20:57:46.890 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T20:57:46.892 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-10T20:57:46.917 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T20:57:46.917 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T20:57:46.958 INFO:teuthology.orchestra.run.smithi098.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-10T20:57:46.971 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-10T20:57:47.051 INFO:teuthology.orchestra.run.smithi098.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-10T20:57:47.063 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T20:57:47.105 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-10T20:57:47.118 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-10T20:57:47.198 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-10T20:57:47.211 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T20:57:47.211 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi098.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T20:57:47.211 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T20:57:47.236 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T20:57:47.236 INFO:tasks.ceph:role: osd.0 2024-09-10T20:57:47.236 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T20:57:47.236 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T20:57:47.308 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T20:57:47.308 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T20:57:47.308 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T20:57:47.308 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T20:57:47.308 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T20:57:47.308 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-09-10T20:57:47.309 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T20:57:47.309 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T20:57:47.309 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T20:57:47.309 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T20:57:47.814 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-09-10T20:57:47.816 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-09-10T20:57:47.816 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T20:57:47.930 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T20:57:47.962 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T20:57:48.027 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T20:57:48.027 INFO:tasks.ceph:role: osd.1 2024-09-10T20:57:48.027 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T20:57:48.027 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T20:57:48.099 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T20:57:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T20:57:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T20:57:48.099 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T20:57:48.099 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T20:57:48.100 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-09-10T20:57:48.100 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T20:57:48.100 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T20:57:48.100 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T20:57:48.100 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T20:57:48.560 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-09-10T20:57:48.561 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-09-10T20:57:48.562 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T20:57:48.649 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T20:57:48.677 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T20:57:48.742 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T20:57:48.743 INFO:tasks.ceph:role: osd.2 2024-09-10T20:57:48.743 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T20:57:48.743 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T20:57:48.816 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T20:57:48.816 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T20:57:48.816 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T20:57:48.816 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T20:57:48.816 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T20:57:48.816 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-09-10T20:57:48.816 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T20:57:48.816 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T20:57:48.817 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T20:57:48.817 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T20:57:49.305 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-09-10T20:57:49.307 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-09-10T20:57:49.307 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T20:57:49.451 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T20:57:49.481 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-10T20:57:49.545 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T20:57:49.545 INFO:tasks.ceph:role: osd.3 2024-09-10T20:57:49.545 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T20:57:49.546 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T20:57:49.618 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T20:57:49.618 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T20:57:49.618 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T20:57:49.618 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T20:57:49.619 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T20:57:49.619 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-09-10T20:57:49.619 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T20:57:49.619 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T20:57:49.619 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T20:57:49.619 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T20:57:50.128 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-09-10T20:57:50.130 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-09-10T20:57:50.131 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-10T20:57:50.262 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-10T20:57:50.294 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:57:50.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:57:50.400+0000 7fb7d480d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T20:57:50.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:57:50.401+0000 7fb7d480d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T20:57:50.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:57:50.401+0000 7fb7d480d640 -1 bdev(0x555e8adf7800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T20:57:50.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:57:50.401+0000 7fb7d480d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T20:57:53.555 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T20:57:53.585 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:57:53.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:57:53.671+0000 7f0d5b61c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T20:57:53.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:57:53.671+0000 7f0d5b61c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T20:57:53.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:57:53.671+0000 7f0d5b61c640 -1 bdev(0x5570537ab800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T20:57:53.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:57:53.671+0000 7f0d5b61c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T20:57:57.073 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T20:57:57.102 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:57:57.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:57:57.188+0000 7f74409a8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T20:57:57.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:57:57.188+0000 7f74409a8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T20:57:57.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:57:57.188+0000 7f74409a8640 -1 bdev(0x55e19c4eb800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T20:57:57.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:57:57.189+0000 7f74409a8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T20:58:00.362 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T20:58:00.390 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:58:00.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:00.475+0000 7f000d01a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-10T20:58:00.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:00.475+0000 7f000d01a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-10T20:58:00.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:00.475+0000 7f000d01a640 -1 bdev(0x558d6dfb5800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-10T20:58:00.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:00.475+0000 7f000d01a640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-10T20:58:03.643 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T20:58:03.672 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-10T20:58:03.703 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T20:58:03.703 INFO:tasks.ceph:role: osd.4 2024-09-10T20:58:03.703 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-10T20:58:03.703 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T20:58:03.778 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T20:58:03.778 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T20:58:03.778 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T20:58:03.778 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T20:58:03.779 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T20:58:03.779 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-09-10T20:58:03.779 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T20:58:03.779 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T20:58:03.779 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T20:58:03.779 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T20:58:04.281 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-09-10T20:58:04.283 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-09-10T20:58:04.284 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-10T20:58:04.426 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-10T20:58:04.458 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-10T20:58:04.523 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T20:58:04.523 INFO:tasks.ceph:role: osd.5 2024-09-10T20:58:04.523 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-10T20:58:04.524 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T20:58:04.596 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T20:58:04.596 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T20:58:04.596 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T20:58:04.596 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T20:58:04.596 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T20:58:04.596 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-09-10T20:58:04.596 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T20:58:04.596 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T20:58:04.597 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T20:58:04.597 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T20:58:05.035 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-09-10T20:58:05.037 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-09-10T20:58:05.038 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-10T20:58:05.125 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-10T20:58:05.153 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-10T20:58:05.218 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T20:58:05.219 INFO:tasks.ceph:role: osd.6 2024-09-10T20:58:05.219 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-10T20:58:05.219 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T20:58:05.292 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T20:58:05.292 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T20:58:05.292 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T20:58:05.292 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T20:58:05.292 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T20:58:05.292 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-09-10T20:58:05.292 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T20:58:05.292 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T20:58:05.292 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T20:58:05.293 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T20:58:05.788 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-09-10T20:58:05.789 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-09-10T20:58:05.790 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-10T20:58:05.906 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-10T20:58:05.935 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-10T20:58:06.000 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T20:58:06.001 INFO:tasks.ceph:role: osd.7 2024-09-10T20:58:06.001 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-10T20:58:06.001 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T20:58:06.072 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T20:58:06.073 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T20:58:06.073 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T20:58:06.073 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T20:58:06.073 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T20:58:06.073 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-09-10T20:58:06.073 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T20:58:06.073 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T20:58:06.073 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T20:58:06.073 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T20:58:06.591 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-09-10T20:58:06.593 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-09-10T20:58:06.593 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-10T20:58:06.721 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-10T20:58:06.751 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:58:06.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:06.851+0000 7f57dbffd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-10T20:58:06.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:06.851+0000 7f57dbffd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-10T20:58:06.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:06.851+0000 7f57dbffd640 -1 bdev(0x563e5c421800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-10T20:58:06.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:06.851+0000 7f57dbffd640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-10T20:58:10.253 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T20:58:10.282 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:58:10.373 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:10.367+0000 7f396ea42640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-10T20:58:10.373 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:10.368+0000 7f396ea42640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-10T20:58:10.373 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:10.368+0000 7f396ea42640 -1 bdev(0x557d21521800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-10T20:58:10.373 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:10.368+0000 7f396ea42640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-10T20:58:13.278 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T20:58:13.308 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:58:13.398 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:13.393+0000 7f19d83bd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-10T20:58:13.399 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:13.394+0000 7f19d83bd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-10T20:58:13.399 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:13.394+0000 7f19d83bd640 -1 bdev(0x5571bed11800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-10T20:58:13.399 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:13.394+0000 7f19d83bd640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-10T20:58:14.040 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T20:58:14.043 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T20:58:16.555 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-10T20:58:16.585 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:58:16.675 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:16.671+0000 7f324dae6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-10T20:58:16.675 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:16.671+0000 7f324dae6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-10T20:58:16.675 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:16.671+0000 7f324dae6640 -1 bdev(0x55e6331b9800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-10T20:58:16.675 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:16.671+0000 7f324dae6640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-10T20:58:19.578 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T20:58:19.607 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T20:58:19.608 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:19.608 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-10T20:58:19.638 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:19.638 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-10T20:58:19.706 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:19.706 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-10T20:58:19.773 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:19.774 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T20:58:19.840 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:19.840 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T20:58:19.907 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:19.907 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T20:58:19.973 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:19.973 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-10T20:58:20.038 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:58:20.038 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T20:58:20.068 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:58:20.068 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-10T20:58:20.136 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:58:20.136 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-10T20:58:20.204 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:58:20.204 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-10T20:58:20.270 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:58:20.270 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-10T20:58:20.336 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:58:20.336 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-10T20:58:20.401 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:58:20.401 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-10T20:58:20.468 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:20.468 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-10T20:58:20.485 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:20.485 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-10T20:58:20.542 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:58:20.543 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-10T20:58:20.559 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:58:20.559 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-10T20:58:20.617 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T20:58:20.617 DEBUG:teuthology.orchestra.run.smithi098:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T20:58:20.619 DEBUG:teuthology.orchestra.run.smithi190:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T20:58:20.659 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr.y] 2024-09-10T20:58:20.659 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDKsuBme6kzGxAAcvRmmGzk/xeEYu9MHDpypg== 2024-09-10T20:58:20.659 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.a] 2024-09-10T20:58:20.660 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDKsuBmDQc7IhAAfVGGH67CSTDxtnhwsoVZjg== 2024-09-10T20:58:20.660 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.c] 2024-09-10T20:58:20.660 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDKsuBmIjk8KRAAM5ZHLlzSDFDzwax1tqAuuw== 2024-09-10T20:58:20.660 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.0] 2024-09-10T20:58:20.660 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDOsuBmsYv0FxAAxF0+GnRAMDFdUFA8NqVuPA== 2024-09-10T20:58:20.660 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.1] 2024-09-10T20:58:20.660 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDRsuBmaiMRKBAAOI/EtX7NA6eaeNwOC/VptA== 2024-09-10T20:58:20.660 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.2] 2024-09-10T20:58:20.660 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDVsuBmDVVMCxAACm/EhpctxJidYZRHKTksCw== 2024-09-10T20:58:20.660 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.3] 2024-09-10T20:58:20.660 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDYsuBmFv9jHBAAGwAe5pgbvheRQaGOgEM70A== 2024-09-10T20:58:20.660 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr.x] 2024-09-10T20:58:20.660 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDKsuBm7OUOHxAANoyF+Xxo2J5mgEShMqV/Rw== 2024-09-10T20:58:20.661 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.b] 2024-09-10T20:58:20.661 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDKsuBmjXnsLRAAmoNy11LPHYxA1o9GX7i90g== 2024-09-10T20:58:20.661 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.d] 2024-09-10T20:58:20.661 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDKsuBmi9PnNBAACkF8eU6DQLO6lfgg/y7BWw== 2024-09-10T20:58:20.661 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.4] 2024-09-10T20:58:20.661 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDesuBmvjnNMhAAxSyqvd3hcOndQZqE7b9Mug== 2024-09-10T20:58:20.661 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.5] 2024-09-10T20:58:20.661 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDisuBmdlELFhAASuvCbhdNG/3eLu8+MUnMpw== 2024-09-10T20:58:20.661 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.6] 2024-09-10T20:58:20.661 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDlsuBmwZ2UFxAA+kl84axYER+Zf74///EPtA== 2024-09-10T20:58:20.661 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.7] 2024-09-10T20:58:20.661 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDosuBmJD0QKBAA/LnYMX4Yt9Y4qgJs6sYErg== 2024-09-10T20:58:20.662 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-10T20:58:20.662 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDKsuBm9hT3OBAAecwpOYTkz+JIO0Xg/GolGw== 2024-09-10T20:58:20.662 INFO:teuthology.orchestra.run.smithi098.stdout:[client.2] 2024-09-10T20:58:20.662 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDLsuBmoKLnAhAArP2MzQowgxjBwOob6Yd0/Q== 2024-09-10T20:58:20.662 INFO:teuthology.orchestra.run.smithi098.stdout:[client.1] 2024-09-10T20:58:20.662 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDLsuBmfGwbBhAA9wRJAg8fHtVSoExyGmo3bw== 2024-09-10T20:58:20.662 INFO:teuthology.orchestra.run.smithi098.stdout:[client.3] 2024-09-10T20:58:20.662 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDLsuBmJoCtCxAAYcwdH65t1Px4BeLbOa15fw== 2024-09-10T20:58:20.680 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.y] 2024-09-10T20:58:20.680 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDKsuBme6kzGxAAcvRmmGzk/xeEYu9MHDpypg== 2024-09-10T20:58:20.680 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2024-09-10T20:58:20.680 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDKsuBmDQc7IhAAfVGGH67CSTDxtnhwsoVZjg== 2024-09-10T20:58:20.680 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2024-09-10T20:58:20.680 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDKsuBmIjk8KRAAM5ZHLlzSDFDzwax1tqAuuw== 2024-09-10T20:58:20.680 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.0] 2024-09-10T20:58:20.680 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDOsuBmsYv0FxAAxF0+GnRAMDFdUFA8NqVuPA== 2024-09-10T20:58:20.680 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.1] 2024-09-10T20:58:20.680 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDRsuBmaiMRKBAAOI/EtX7NA6eaeNwOC/VptA== 2024-09-10T20:58:20.681 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.2] 2024-09-10T20:58:20.681 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDVsuBmDVVMCxAACm/EhpctxJidYZRHKTksCw== 2024-09-10T20:58:20.681 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.3] 2024-09-10T20:58:20.681 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDYsuBmFv9jHBAAGwAe5pgbvheRQaGOgEM70A== 2024-09-10T20:58:20.681 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.x] 2024-09-10T20:58:20.681 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDKsuBm7OUOHxAANoyF+Xxo2J5mgEShMqV/Rw== 2024-09-10T20:58:20.681 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2024-09-10T20:58:20.681 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDKsuBmjXnsLRAAmoNy11LPHYxA1o9GX7i90g== 2024-09-10T20:58:20.681 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2024-09-10T20:58:20.681 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDKsuBmi9PnNBAACkF8eU6DQLO6lfgg/y7BWw== 2024-09-10T20:58:20.681 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.4] 2024-09-10T20:58:20.681 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDesuBmvjnNMhAAxSyqvd3hcOndQZqE7b9Mug== 2024-09-10T20:58:20.681 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.5] 2024-09-10T20:58:20.681 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDisuBmdlELFhAASuvCbhdNG/3eLu8+MUnMpw== 2024-09-10T20:58:20.682 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.6] 2024-09-10T20:58:20.682 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDlsuBmwZ2UFxAA+kl84axYER+Zf74///EPtA== 2024-09-10T20:58:20.682 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.7] 2024-09-10T20:58:20.682 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDosuBmJD0QKBAA/LnYMX4Yt9Y4qgJs6sYErg== 2024-09-10T20:58:20.682 INFO:teuthology.orchestra.run.smithi190.stdout:[client.0] 2024-09-10T20:58:20.682 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDKsuBm9hT3OBAAecwpOYTkz+JIO0Xg/GolGw== 2024-09-10T20:58:20.682 INFO:teuthology.orchestra.run.smithi190.stdout:[client.2] 2024-09-10T20:58:20.682 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDLsuBmoKLnAhAArP2MzQowgxjBwOob6Yd0/Q== 2024-09-10T20:58:20.682 INFO:teuthology.orchestra.run.smithi190.stdout:[client.1] 2024-09-10T20:58:20.682 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDLsuBmfGwbBhAA9wRJAg8fHtVSoExyGmo3bw== 2024-09-10T20:58:20.682 INFO:teuthology.orchestra.run.smithi190.stdout:[client.3] 2024-09-10T20:58:20.682 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQDLsuBmJoCtCxAAYcwdH65t1Px4BeLbOa15fw== 2024-09-10T20:58:20.683 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T20:58:20.705 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T20:58:20.776 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T20:58:20.779 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T20:58:20.833 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T20:58:20.836 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T20:58:20.889 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:20.903 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:20.953 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:20.956 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:21.007 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:21.009 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:21.059 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:21.062 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:21.113 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T20:58:21.116 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T20:58:21.166 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T20:58:21.169 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T20:58:21.223 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T20:58:21.226 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T20:58:21.279 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:21.281 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:21.332 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:21.334 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:21.385 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:21.387 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:21.438 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:21.440 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T20:58:21.491 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T20:58:21.493 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T20:58:21.546 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T20:58:21.548 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T20:58:21.601 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T20:58:21.604 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T20:58:21.657 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T20:58:21.660 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T20:58:21.714 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T20:58:21.714 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T20:58:21.738 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T20:58:21.833 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T20:58:21.857 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T20:58:21.882 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T20:58:21.979 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T20:58:22.003 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T20:58:22.068 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T20:58:22.165 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T20:58:22.193 DEBUG:teuthology.orchestra.run.smithi098:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:58:22.196 DEBUG:teuthology.orchestra.run.smithi190:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T20:58:22.250 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T20:58:22.250 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T20:58:22.250 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T20:58:22.254 INFO:tasks.ceph.mon.a:Started 2024-09-10T20:58:22.254 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T20:58:22.254 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T20:58:22.292 INFO:tasks.ceph.mon.b:Started 2024-09-10T20:58:22.292 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T20:58:22.293 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T20:58:22.295 INFO:tasks.ceph.mon.c:Started 2024-09-10T20:58:22.295 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T20:58:22.295 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-10T20:58:22.295 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-10T20:58:22.298 INFO:tasks.ceph.mgr.y:Started 2024-09-10T20:58:22.298 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T20:58:22.298 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T20:58:22.300 INFO:tasks.ceph.mgr.x:Started 2024-09-10T20:58:22.300 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-10T20:58:22.300 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-10T20:58:22.303 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-10T20:58:22.303 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-10T20:58:22.347 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-10T20:58:22.347 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-10T20:58:22.391 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-10T20:58:22.391 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-10T20:58:22.434 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-10T20:58:22.434 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-10T20:58:22.476 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-10T20:58:22.477 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-10T20:58:22.522 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-10T20:58:22.522 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-10T20:58:22.565 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-10T20:58:22.565 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-10T20:58:22.608 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-10T20:58:22.608 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-10T20:58:22.651 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-10T20:58:22.651 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-10T20:58:22.694 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-10T20:58:22.694 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-10T20:58:22.740 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-10T20:58:22.740 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-10T20:58:22.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.739+0000 7f3bbf4e2640 1 -- 172.21.15.98:0/278591022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ba40037a0 msgr2=0x7f3bb80cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:22.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.739+0000 7f3bbf4e2640 1 --2- 172.21.15.98:0/278591022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ba40037a0 0x7f3bb80cc3c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006f70 tx=0x7f3bb40039c0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.739+0000 7f3bbf4e2640 1 -- 172.21.15.98:0/278591022 shutdown_connections 2024-09-10T20:58:22.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.739+0000 7f3bbf4e2640 1 --2- 172.21.15.98:0/278591022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bb8099110 0x7f3bb80cfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.744 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-10T20:58:22.744 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-10T20:58:22.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.739+0000 7f3bbf4e2640 1 --2- 172.21.15.98:0/278591022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bb80cf5a0 0x7f3bb8098bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.739+0000 7f3bbf4e2640 1 --2- 172.21.15.98:0/278591022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ba40037a0 0x7f3bb80cc3c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.739+0000 7f3bbf4e2640 1 -- 172.21.15.98:0/278591022 >> 172.21.15.98:0/278591022 conn(0x7f3bb8074190 msgr2=0x7f3bb80c5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:22.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.740+0000 7f3bbf4e2640 1 -- 172.21.15.98:0/278591022 shutdown_connections 2024-09-10T20:58:22.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.740+0000 7f3bbf4e2640 1 -- 172.21.15.98:0/278591022 wait complete. 2024-09-10T20:58:22.749 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T20:58:22.749 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-10T20:58:22.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.749+0000 7f3652794640 1 -- 172.21.15.98:0/4157030281 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f36240052f0 con 0x7f3634003a50 2024-09-10T20:58:22.755 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T20:58:22.756 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-10T20:58:22.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.752+0000 7f99f2175640 1 -- 172.21.15.98:0/825529875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99ec0b2fa0 msgr2=0x7f99ec072b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.754+0000 7f3648ff9640 1 -- 172.21.15.98:0/4157030281 <== mon.1 v2:172.21.15.190:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f363c047870 con 0x7f3634003a50 2024-09-10T20:58:22.759 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-10T20:58:22.760 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-10T20:58:22.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.755+0000 7f3648ff9640 1 -- 172.21.15.98:0/4157030281 <== mon.1 v2:172.21.15.190:3300/0 11 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v5)=0 v5) ==== 114+0+0 (secure 0 0 0) 0x7f363c047b50 con 0x7f3634003a50 2024-09-10T20:58:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.757+0000 7f23c8ff9640 1 -- 172.21.15.98:0/3670938285 <== mon.0 v2:172.21.15.98:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f23bc00f660 con 0x7f23b40051c0 2024-09-10T20:58:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.757+0000 7f36327fc640 1 -- 172.21.15.98:0/4157030281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3634003a50 msgr2=0x7f364c1efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.757+0000 7f36327fc640 1 --2- 172.21.15.98:0/4157030281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3634003a50 0x7f364c1efc60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f363c00dc10 tx=0x7f363c00c270 comp rx=0 tx=0).stop 2024-09-10T20:58:22.762 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T20:58:22.762 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T20:58:22.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.758+0000 7f36327fc640 1 -- 172.21.15.98:0/4157030281 shutdown_connections 2024-09-10T20:58:22.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.752+0000 7f99f2175640 1 --2- 172.21.15.98:0/825529875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99ec0b2fa0 0x7f99ec072b90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f99ec069cf0 tx=0x7f99e402aa50 comp rx=0 tx=0).stop 2024-09-10T20:58:22.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.759+0000 7f99f2175640 1 -- 172.21.15.98:0/825529875 shutdown_connections 2024-09-10T20:58:22.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.759+0000 7f99f2175640 1 --2- 172.21.15.98:0/825529875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99ec0b2fa0 0x7f99ec072b90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.759+0000 7f99f2175640 1 --2- 172.21.15.98:0/825529875 >> v2:172.21.15.190:3300/0 conn(0x7f99ec0ad890 0x7f99ec0b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.759+0000 7f99f2175640 1 --2- 172.21.15.98:0/825529875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99ec1477b0 0x7f99ec0b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.759+0000 7f99f2175640 1 -- 172.21.15.98:0/825529875 >> 172.21.15.98:0/825529875 conn(0x7f99ec0cfaf0 msgr2=0x7f99ec0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:22.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.765+0000 7f36327fc640 1 --2- 172.21.15.98:0/4157030281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f364c1f2dc0 0x7f364c06ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.765+0000 7f36327fc640 1 --2- 172.21.15.98:0/4157030281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f364c153470 0x7f364c1f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.765+0000 7f36327fc640 1 --2- 172.21.15.98:0/4157030281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3634003a50 0x7f364c1efc60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.765+0000 7f36327fc640 1 -- 172.21.15.98:0/4157030281 >> 172.21.15.98:0/4157030281 conn(0x7f364c074190 msgr2=0x7f364c1566d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:22.774 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-10T20:58:22.774 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-10T20:58:22.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.765+0000 7f99f2175640 1 -- 172.21.15.98:0/825529875 shutdown_connections 2024-09-10T20:58:22.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.765+0000 7f99f2175640 1 -- 172.21.15.98:0/825529875 wait complete. 2024-09-10T20:58:22.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.767+0000 7f99f2175640 1 Processor -- start 2024-09-10T20:58:22.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.768+0000 7f99f2175640 1 -- start start 2024-09-10T20:58:22.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.770+0000 7f99f2175640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99ec0ad890 0x7f99ec1f9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:22.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.770+0000 7f99f2175640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99ec0b2fa0 0x7f99ec15d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:22.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.770+0000 7f99f2175640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99ec1477b0 0x7f99ec0aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:22.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.770+0000 7f99eb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99ec0ad890 0x7f99ec1f9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:22.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.770+0000 7f99eb7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99ec0ad890 0x7f99ec1f9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:60668/0 (socket says 172.21.15.98:60668) 2024-09-10T20:58:22.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.770+0000 7f99eaffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99ec0b2fa0 0x7f99ec15d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:22.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.770+0000 7f99f2175640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec073460 con 0x7f99ec0b2fa0 2024-09-10T20:58:22.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.770+0000 7f99eaffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99ec0b2fa0 0x7f99ec15d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49628/0 (socket says 172.21.15.98:49628) 2024-09-10T20:58:22.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.770+0000 7f99eaffd640 1 -- 172.21.15.98:0/4062530651 learned_addr learned my addr 172.21.15.98:0/4062530651 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:22.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.770+0000 7f99f2175640 1 -- 172.21.15.98:0/4062530651 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec0732e0 con 0x7f99ec0ad890 2024-09-10T20:58:22.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.770+0000 7f99f2175640 1 -- 172.21.15.98:0/4062530651 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f99ec0735e0 con 0x7f99ec1477b0 2024-09-10T20:58:22.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.773+0000 7fc2bcff9640 1 -- 172.21.15.98:0/2337903332 <== mon.0 v2:172.21.15.98:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fc2b0004bb0 con 0x7fc2c01582b0 2024-09-10T20:58:22.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.773+0000 7f99ebfff640 1 --2- 172.21.15.98:0/4062530651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99ec1477b0 0x7f99ec0aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:22.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.773+0000 7f99eb7fe640 1 -- 172.21.15.98:0/4062530651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99ec1477b0 msgr2=0x7f99ec0aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:22.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.773+0000 7f99eb7fe640 1 --2- 172.21.15.98:0/4062530651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99ec1477b0 0x7f99ec0aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.773+0000 7f99eb7fe640 1 -- 172.21.15.98:0/4062530651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99ec0b2fa0 msgr2=0x7f99ec15d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:22.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.773+0000 7f99eb7fe640 1 --2- 172.21.15.98:0/4062530651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99ec0b2fa0 0x7f99ec15d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.773+0000 7f99eb7fe640 1 -- 172.21.15.98:0/4062530651 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ec0a99d0 con 0x7f99ec0ad890 2024-09-10T20:58:22.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.773+0000 7f99eb7fe640 1 --2- 172.21.15.98:0/4062530651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99ec0ad890 0x7f99ec1f9410 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f99e000b3e0 tx=0x7f99e000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:22.781 DEBUG:tasks.ceph:set 18 configs 2024-09-10T20:58:22.781 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi098') 2024-09-10T20:58:22.782 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi098') 2024-09-10T20:58:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.776+0000 7f99d3fff640 1 -- 172.21.15.98:0/4062530651 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0015020 con 0x7f99ec0ad890 2024-09-10T20:58:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.776+0000 7f99d3fff640 1 -- 172.21.15.98:0/4062530651 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f99e0011900 con 0x7f99ec0ad890 2024-09-10T20:58:22.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.776+0000 7f99f2175640 1 -- 172.21.15.98:0/4062530651 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99ec0a9cb0 con 0x7f99ec0ad890 2024-09-10T20:58:22.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.776+0000 7f99f2175640 1 -- 172.21.15.98:0/4062530651 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99ec0aa270 con 0x7f99ec0ad890 2024-09-10T20:58:22.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.777+0000 7f36327fc640 1 -- 172.21.15.98:0/4157030281 shutdown_connections 2024-09-10T20:58:22.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.777+0000 7f36327fc640 1 -- 172.21.15.98:0/4157030281 wait complete. 2024-09-10T20:58:22.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.777+0000 7f99d3fff640 1 -- 172.21.15.98:0/4062530651 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0011cf0 con 0x7f99ec0ad890 2024-09-10T20:58:22.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.777+0000 7f99d3fff640 1 -- 172.21.15.98:0/4062530651 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f99e001a790 con 0x7f99ec0ad890 2024-09-10T20:58:22.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.778+0000 7f99d3fff640 1 -- 172.21.15.98:0/4062530651 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f99e0019c90 con 0x7f99ec0ad890 2024-09-10T20:58:22.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.779+0000 7f99d1ffb640 1 -- 172.21.15.98:0/4062530651 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99cc005290 con 0x7f99ec0ad890 2024-09-10T20:58:22.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.782+0000 7f99d3fff640 1 -- 172.21.15.98:0/4062530651 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f99e001f020 con 0x7f99ec0ad890 2024-09-10T20:58:22.807 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi098') 2024-09-10T20:58:22.807 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi098') 2024-09-10T20:58:22.807 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi098') 2024-09-10T20:58:22.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.814+0000 7f23b27fc640 1 -- 172.21.15.98:0/3670938285 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f23a4005310 con 0x7f23b40051c0 2024-09-10T20:58:22.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.829+0000 7fc2bcff9640 1 -- 172.21.15.98:0/2337903332 <== mon.0 v2:172.21.15.98:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2b0004dd0 con 0x7fc2c01582b0 2024-09-10T20:58:22.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.830+0000 7f23c8ff9640 1 -- 172.21.15.98:0/3670938285 <== mon.0 v2:172.21.15.98:3300/0 11 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f23bc00f9e0 con 0x7f23b40051c0 2024-09-10T20:58:22.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.830+0000 7f23c8ff9640 1 -- 172.21.15.98:0/3670938285 <== mon.0 v2:172.21.15.98:3300/0 12 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23bc00fcc0 con 0x7f23b40051c0 2024-09-10T20:58:22.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.838+0000 7f99d3fff640 1 -- 172.21.15.98:0/4062530651 <== mon.1 v2:172.21.15.190:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99e00193b0 con 0x7f99ec0ad890 2024-09-10T20:58:22.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.838+0000 7f23d275d640 1 -- 172.21.15.98:0/3670938285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23b40051c0 msgr2=0x7f23cc15adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:22.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.838+0000 7f23d275d640 1 --2- 172.21.15.98:0/3670938285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23b40051c0 0x7f23cc15adc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f23bc000f60 tx=0x7f23bc00cc90 comp rx=0 tx=0).stop 2024-09-10T20:58:22.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.842+0000 7fb8e10b6640 1 -- 172.21.15.98:0/760020339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8dc0cf400 msgr2=0x7fb8dc0738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:22.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.842+0000 7fb8e10b6640 1 --2- 172.21.15.98:0/760020339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8dc0cf400 0x7fb8dc0738e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0b5cb0 tx=0x7fb8cc02ea10 comp rx=0 tx=0).stop 2024-09-10T20:58:22.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.844+0000 7fb8e10b6640 1 -- 172.21.15.98:0/760020339 shutdown_connections 2024-09-10T20:58:22.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.844+0000 7fb8e10b6640 1 --2- 172.21.15.98:0/760020339 >> v2:172.21.15.98:3300/0 conn(0x7fb8dc16e1f0 0x7fb8dc06fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.844+0000 7fb8e10b6640 1 --2- 172.21.15.98:0/760020339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8dc0723b0 0x7fb8dc06f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.844+0000 7fb8e10b6640 1 --2- 172.21.15.98:0/760020339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8dc0cf400 0x7fb8dc0738e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.844+0000 7fb8e10b6640 1 -- 172.21.15.98:0/760020339 >> 172.21.15.98:0/760020339 conn(0x7fb8dc0af660 msgr2=0x7fb8dc0afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:22.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.847+0000 7f23d275d640 1 -- 172.21.15.98:0/3670938285 shutdown_connections 2024-09-10T20:58:22.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.847+0000 7f23d275d640 1 --2- 172.21.15.98:0/3670938285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23cc0af600 0x7f23cc08b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.847+0000 7f23d275d640 1 --2- 172.21.15.98:0/3670938285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23cc0ad240 0x7f23cc0a9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.847+0000 7f23d275d640 1 --2- 172.21.15.98:0/3670938285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23b40051c0 0x7f23cc15adc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.847+0000 7f23d275d640 1 -- 172.21.15.98:0/3670938285 >> 172.21.15.98:0/3670938285 conn(0x7f23cc06e480 msgr2=0x7f23cc15d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:22.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.845+0000 7fb8e10b6640 1 -- 172.21.15.98:0/760020339 shutdown_connections 2024-09-10T20:58:22.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.845+0000 7fb8e10b6640 1 -- 172.21.15.98:0/760020339 wait complete. 2024-09-10T20:58:22.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.847+0000 7fb8e10b6640 1 Processor -- start 2024-09-10T20:58:22.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.847+0000 7fb8e10b6640 1 -- start start 2024-09-10T20:58:22.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.848+0000 7fb8e10b6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8dc0cf400 0x7fb8dc155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:22.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.848+0000 7fb8e10b6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8dc16e1f0 0x7fb8dc166c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:22.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.848+0000 7fb8e10b6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8dc167140 0x7fb8dc16ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:22.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.848+0000 7fb8e10b6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc0cce00 con 0x7fb8dc167140 2024-09-10T20:58:22.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.848+0000 7fb8e10b6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc0ccc80 con 0x7fb8dc16e1f0 2024-09-10T20:58:22.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.848+0000 7fb8e10b6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb8dc0ccf80 con 0x7fb8dc0cf400 2024-09-10T20:58:22.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.849+0000 7fb8da59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8dc16e1f0 0x7fb8dc166c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:22.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.849+0000 7fb8dad9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8dc0cf400 0x7fb8dc155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:22.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.849+0000 7fb8da59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8dc16e1f0 0x7fb8dc166c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:60684/0 (socket says 172.21.15.98:60684) 2024-09-10T20:58:22.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.849+0000 7fb8da59c640 1 -- 172.21.15.98:0/988092118 learned_addr learned my addr 172.21.15.98:0/988092118 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:22.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.849+0000 7fb8db59e640 1 --2- 172.21.15.98:0/988092118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8dc167140 0x7fb8dc16ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:22.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.849+0000 7fb8da59c640 1 -- 172.21.15.98:0/988092118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8dc0cf400 msgr2=0x7fb8dc155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:22.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.849+0000 7fb8da59c640 1 --2- 172.21.15.98:0/988092118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8dc0cf400 0x7fb8dc155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.849+0000 7fb8da59c640 1 -- 172.21.15.98:0/988092118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8dc167140 msgr2=0x7fb8dc16ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:22.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.849+0000 7fb8da59c640 1 --2- 172.21.15.98:0/988092118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8dc167140 0x7fb8dc16ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.849+0000 7fb8da59c640 1 -- 172.21.15.98:0/988092118 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc155fc0 con 0x7fb8dc16e1f0 2024-09-10T20:58:22.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.849+0000 7fb8db59e640 1 --2- 172.21.15.98:0/988092118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8dc167140 0x7fb8dc16ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:58:22.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.851+0000 7fb8da59c640 1 --2- 172.21.15.98:0/988092118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8dc16e1f0 0x7fb8dc166c00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400efc0 tx=0x7fb8d400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:22.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.856+0000 7fb8c37fe640 1 -- 172.21.15.98:0/988092118 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4004b20 con 0x7fb8dc16e1f0 2024-09-10T20:58:22.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.856+0000 7fb8e10b6640 1 -- 172.21.15.98:0/988092118 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8dc204be0 con 0x7fb8dc16e1f0 2024-09-10T20:58:22.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.856+0000 7fb8e10b6640 1 -- 172.21.15.98:0/988092118 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8dc205460 con 0x7fb8dc16e1f0 2024-09-10T20:58:22.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.857+0000 7fb8c37fe640 1 -- 172.21.15.98:0/988092118 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8d4004db0 con 0x7fb8dc16e1f0 2024-09-10T20:58:22.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.857+0000 7fb8c37fe640 1 -- 172.21.15.98:0/988092118 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d400b890 con 0x7fb8dc16e1f0 2024-09-10T20:58:22.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.858+0000 7fb8c37fe640 1 -- 172.21.15.98:0/988092118 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fb8d400b040 con 0x7fb8dc16e1f0 2024-09-10T20:58:22.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.858+0000 7fb8c37fe640 1 -- 172.21.15.98:0/988092118 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb8d4005840 con 0x7fb8dc16e1f0 2024-09-10T20:58:22.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.859+0000 7fb8e10b6640 1 -- 172.21.15.98:0/988092118 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8bc005290 con 0x7fb8dc16e1f0 2024-09-10T20:58:22.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.861+0000 7f23d275d640 1 -- 172.21.15.98:0/3670938285 shutdown_connections 2024-09-10T20:58:22.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.861+0000 7f23d275d640 1 -- 172.21.15.98:0/3670938285 wait complete. 2024-09-10T20:58:22.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.861+0000 7fb8c37fe640 1 -- 172.21.15.98:0/988092118 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fb8dc155fc0 con 0x7fb8dc16e1f0 2024-09-10T20:58:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.893+0000 7fd3fea03640 1 -- 172.21.15.98:0/3441233386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3f80723b0 msgr2=0x7fd3f806f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.893+0000 7fd3fea03640 1 --2- 172.21.15.98:0/3441233386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3f80723b0 0x7fd3f806f600 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0005fd0 tx=0x7fd3f002cbd0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.893+0000 7fd3fea03640 1 -- 172.21.15.98:0/3441233386 shutdown_connections 2024-09-10T20:58:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.893+0000 7fd3fea03640 1 --2- 172.21.15.98:0/3441233386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3f80723b0 0x7fd3f806f600 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.893+0000 7fd3fea03640 1 --2- 172.21.15.98:0/3441233386 >> v2:172.21.15.98:3300/0 conn(0x7fd3f80cf400 0x7fd3f80738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.893+0000 7fd3fea03640 1 -- 172.21.15.98:0/3441233386 >> 172.21.15.98:0/3441233386 conn(0x7fd3f80af660 msgr2=0x7fd3f80afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:22.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.893+0000 7fd3fea03640 1 -- 172.21.15.98:0/3441233386 shutdown_connections 2024-09-10T20:58:22.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.893+0000 7fd3fea03640 1 -- 172.21.15.98:0/3441233386 wait complete. 2024-09-10T20:58:22.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.895+0000 7fd3fea03640 1 Processor -- start 2024-09-10T20:58:22.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.896+0000 7fd3fea03640 1 -- start start 2024-09-10T20:58:22.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.897+0000 7fd3fea03640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3f80cf400 0x7fd3f8089b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:22.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.897+0000 7fd3fea03640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f816e1f0 0x7fd3f8169b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:22.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.897+0000 7fd3fea03640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3f80c51c0 0x7fd3f80b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:22.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.897+0000 7fd3fea03640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f80cd180 con 0x7fd3f816e1f0 2024-09-10T20:58:22.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.897+0000 7fd3fea03640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f806fe60 con 0x7fd3f80cf400 2024-09-10T20:58:22.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.897+0000 7fd3fea03640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f80cd300 con 0x7fd3f80c51c0 2024-09-10T20:58:22.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.897+0000 7fd3f7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3f80cf400 0x7fd3f8089b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:22.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.897+0000 7fd3f7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3f80cf400 0x7fd3f8089b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:60688/0 (socket says 172.21.15.98:60688) 2024-09-10T20:58:22.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.897+0000 7fd3f7fff640 1 -- 172.21.15.98:0/236264388 learned_addr learned my addr 172.21.15.98:0/236264388 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:22.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.897+0000 7fd3f77fe640 1 --2- 172.21.15.98:0/236264388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f816e1f0 0x7fd3f8169b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:22.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.898+0000 7fd3f7fff640 1 -- 172.21.15.98:0/236264388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3f80c51c0 msgr2=0x7fd3f80b2b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T20:58:22.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.898+0000 7fd3f7fff640 1 --2- 172.21.15.98:0/236264388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3f80c51c0 0x7fd3f80b2b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.898+0000 7fd3f7fff640 1 -- 172.21.15.98:0/236264388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f816e1f0 msgr2=0x7fd3f8169b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:22.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.898+0000 7fd3f7fff640 1 --2- 172.21.15.98:0/236264388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f816e1f0 0x7fd3f8169b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.898+0000 7fd3f7fff640 1 -- 172.21.15.98:0/236264388 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f8168aa0 con 0x7fd3f80cf400 2024-09-10T20:58:22.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.898+0000 7fd3f77fe640 1 --2- 172.21.15.98:0/236264388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f816e1f0 0x7fd3f8169b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:58:22.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.898+0000 7fd3f7fff640 1 --2- 172.21.15.98:0/236264388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3f80cf400 0x7fd3f8089b90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800fba0 tx=0x7fd3e800d7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:22.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.899+0000 7fd3f4ff9640 1 -- 172.21.15.98:0/236264388 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8004220 con 0x7fd3f80cf400 2024-09-10T20:58:22.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.900+0000 7fd3fea03640 1 -- 172.21.15.98:0/236264388 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f8168d80 con 0x7fd3f80cf400 2024-09-10T20:58:22.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.900+0000 7fd3fea03640 1 -- 172.21.15.98:0/236264388 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f8169340 con 0x7fd3f80cf400 2024-09-10T20:58:22.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.901+0000 7fd3f4ff9640 1 -- 172.21.15.98:0/236264388 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3e80043c0 con 0x7fd3f80cf400 2024-09-10T20:58:22.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.901+0000 7fd3f4ff9640 1 -- 172.21.15.98:0/236264388 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8003850 con 0x7fd3f80cf400 2024-09-10T20:58:22.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.901+0000 7fd3f4ff9640 1 -- 172.21.15.98:0/236264388 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fd3e8016070 con 0x7fd3f80cf400 2024-09-10T20:58:22.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.901+0000 7fd3f4ff9640 1 -- 172.21.15.98:0/236264388 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd3e8020780 con 0x7fd3f80cf400 2024-09-10T20:58:22.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.903+0000 7fd3fea03640 1 -- 172.21.15.98:0/236264388 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d8005290 con 0x7fd3f80cf400 2024-09-10T20:58:22.909 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi098') 2024-09-10T20:58:22.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.910+0000 7fd3f4ff9640 1 -- 172.21.15.98:0/236264388 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fd3e80209d0 con 0x7fd3f80cf400 2024-09-10T20:58:22.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.936+0000 7fc2a67fc640 1 -- 172.21.15.98:0/2337903332 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fc2980052f0 con 0x7fc2c01582b0 2024-09-10T20:58:22.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.946+0000 7fc2bcff9640 1 -- 172.21.15.98:0/2337903332 <== mon.0 v2:172.21.15.98:3300/0 12 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7fc2b00414f0 con 0x7fc2c01582b0 2024-09-10T20:58:22.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.952+0000 7fc2a67fc640 1 -- 172.21.15.98:0/2337903332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2c01582b0 msgr2=0x7fc2c0141a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:22.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.952+0000 7fc2a67fc640 1 --2- 172.21.15.98:0/2337903332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2c01582b0 0x7fc2c0141a80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0003d80 tx=0x7fc2b0001e70 comp rx=0 tx=0).stop 2024-09-10T20:58:22.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.952+0000 7fc2a67fc640 1 -- 172.21.15.98:0/2337903332 shutdown_connections 2024-09-10T20:58:22.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.952+0000 7fc2a67fc640 1 --2- 172.21.15.98:0/2337903332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2c0158690 0x7fc2c01409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.952+0000 7fc2a67fc640 1 --2- 172.21.15.98:0/2337903332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2c0158c90 0x7fc2c0140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.952+0000 7fc2a67fc640 1 --2- 172.21.15.98:0/2337903332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2c01582b0 0x7fc2c0141a80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.952+0000 7fc2a67fc640 1 -- 172.21.15.98:0/2337903332 >> 172.21.15.98:0/2337903332 conn(0x7fc2c00af660 msgr2=0x7fc2c00afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:22.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.952+0000 7fc2a67fc640 1 -- 172.21.15.98:0/2337903332 shutdown_connections 2024-09-10T20:58:22.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.952+0000 7fc2a67fc640 1 -- 172.21.15.98:0/2337903332 wait complete. 2024-09-10T20:58:22.981 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi098') 2024-09-10T20:58:22.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.993+0000 7f411a639640 1 -- 172.21.15.98:0/3984091663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41140d0a60 msgr2=0x7f41140af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:22.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.993+0000 7f411a639640 1 --2- 172.21.15.98:0/3984091663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41140d0a60 0x7f41140af600 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f411409e470 tx=0x7f410402a4b0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.993+0000 7f411a639640 1 -- 172.21.15.98:0/3984091663 shutdown_connections 2024-09-10T20:58:22.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.993+0000 7f411a639640 1 --2- 172.21.15.98:0/3984091663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f411406cc60 0x7f411406d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.993+0000 7f411a639640 1 --2- 172.21.15.98:0/3984091663 >> v2:172.21.15.190:3300/0 conn(0x7f41140b59e0 0x7f41140b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.993+0000 7f411a639640 1 --2- 172.21.15.98:0/3984091663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41140d0a60 0x7f41140af600 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:22.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.993+0000 7f411a639640 1 -- 172.21.15.98:0/3984091663 >> 172.21.15.98:0/3984091663 conn(0x7f4114074170 msgr2=0x7f4114074580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.996+0000 7f411a639640 1 -- 172.21.15.98:0/3984091663 shutdown_connections 2024-09-10T20:58:23.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.996+0000 7f411a639640 1 -- 172.21.15.98:0/3984091663 wait complete. 2024-09-10T20:58:23.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.998+0000 7f411a639640 1 Processor -- start 2024-09-10T20:58:23.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.998+0000 7f411a639640 1 -- start start 2024-09-10T20:58:23.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.999+0000 7f411a639640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f411406cc60 0x7f41140c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.999+0000 7f411a639640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41140b59e0 0x7f41140c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.999+0000 7f411a639640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41140d0a60 0x7f4114204c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.999+0000 7f411a639640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f41140ad120 con 0x7f41140b59e0 2024-09-10T20:58:23.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.999+0000 7f411a639640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f41140ad2a0 con 0x7f411406cc60 2024-09-10T20:58:23.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.999+0000 7f411a639640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f41140ad420 con 0x7f41140d0a60 2024-09-10T20:58:23.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.999+0000 7f41137fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41140b59e0 0x7f41140c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.999+0000 7f41137fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41140b59e0 0x7f41140c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49692/0 (socket says 172.21.15.98:49692) 2024-09-10T20:58:23.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:22.999+0000 7f41137fe640 1 -- 172.21.15.98:0/1402821046 learned_addr learned my addr 172.21.15.98:0/1402821046 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:23.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.000+0000 7f41137fe640 1 -- 172.21.15.98:0/1402821046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41140d0a60 msgr2=0x7f4114204c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.000+0000 7f41137fe640 1 --2- 172.21.15.98:0/1402821046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41140d0a60 0x7f4114204c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.000+0000 7f41137fe640 1 -- 172.21.15.98:0/1402821046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f411406cc60 msgr2=0x7f41140c2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.000+0000 7f41137fe640 1 --2- 172.21.15.98:0/1402821046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f411406cc60 0x7f41140c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.000+0000 7f41137fe640 1 -- 172.21.15.98:0/1402821046 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41140c6d40 con 0x7f41140b59e0 2024-09-10T20:58:23.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.000+0000 7f41137fe640 1 --2- 172.21.15.98:0/1402821046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41140b59e0 0x7f41140c6360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0309d0 tx=0x7f40fc030e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:23.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.003+0000 7f4110ff9640 1 -- 172.21.15.98:0/1402821046 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc02ecd0 con 0x7f41140b59e0 2024-09-10T20:58:23.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.003+0000 7f4110ff9640 1 -- 172.21.15.98:0/1402821046 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40fc005020 con 0x7f41140b59e0 2024-09-10T20:58:23.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.003+0000 7f411a639640 1 -- 172.21.15.98:0/1402821046 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41140c7020 con 0x7f41140b59e0 2024-09-10T20:58:23.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.003+0000 7f411a639640 1 -- 172.21.15.98:0/1402821046 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41140cbec0 con 0x7f41140b59e0 2024-09-10T20:58:23.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.005+0000 7f4110ff9640 1 -- 172.21.15.98:0/1402821046 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc005450 con 0x7f41140b59e0 2024-09-10T20:58:23.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.006+0000 7f4110ff9640 1 -- 172.21.15.98:0/1402821046 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f40fc0047a0 con 0x7f41140b59e0 2024-09-10T20:58:23.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.006+0000 7f4110ff9640 1 -- 172.21.15.98:0/1402821046 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f40fc043d70 con 0x7f41140b59e0 2024-09-10T20:58:23.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.006+0000 7f410a7fc640 1 -- 172.21.15.98:0/1402821046 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f8005290 con 0x7f41140b59e0 2024-09-10T20:58:23.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.010+0000 7f4110ff9640 1 -- 172.21.15.98:0/1402821046 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f40fc0055f0 con 0x7f41140b59e0 2024-09-10T20:58:23.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.027+0000 7f6137c5a640 1 -- 172.21.15.98:0/2233675838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f613006fd10 msgr2=0x7f61300738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.027+0000 7f6137c5a640 1 --2- 172.21.15.98:0/2233675838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f613006fd10 0x7f61300738e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6120009880 tx=0x7f612002a3d0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.027+0000 7f6137c5a640 1 -- 172.21.15.98:0/2233675838 shutdown_connections 2024-09-10T20:58:23.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.027+0000 7f6137c5a640 1 --2- 172.21.15.98:0/2233675838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f613006fd10 0x7f61300738e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.027+0000 7f6137c5a640 1 -- 172.21.15.98:0/2233675838 >> 172.21.15.98:0/2233675838 conn(0x7f61300af660 msgr2=0x7f61300afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.034+0000 7f6137c5a640 1 -- 172.21.15.98:0/2233675838 shutdown_connections 2024-09-10T20:58:23.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.041+0000 7fe97b565640 1 -- 172.21.15.98:0/630055092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9740b2cd0 msgr2=0x7fe9740b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.034+0000 7f6137c5a640 1 -- 172.21.15.98:0/2233675838 wait complete. 2024-09-10T20:58:23.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.037+0000 7f6137c5a640 1 Processor -- start 2024-09-10T20:58:23.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.038+0000 7f6137c5a640 1 -- start start 2024-09-10T20:58:23.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.039+0000 7f6137c5a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61301f7270 0x7f61301f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.039+0000 7f6137c5a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61301f7650 0x7f613016c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.039+0000 7f6137c5a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f613016c760 0x7f61302016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.039+0000 7f6137c5a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f61300cd300 con 0x7f613016c760 2024-09-10T20:58:23.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.039+0000 7f6137c5a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f61300cd180 con 0x7f61301f7270 2024-09-10T20:58:23.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.039+0000 7f6137c5a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f61300cd480 con 0x7f61301f7650 2024-09-10T20:58:23.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.039+0000 7f61361f7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f613016c760 0x7f61302016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.040+0000 7f61361f7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f613016c760 0x7f61302016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49728/0 (socket says 172.21.15.98:49728) 2024-09-10T20:58:23.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.040+0000 7f61361f7640 1 -- 172.21.15.98:0/3592259835 learned_addr learned my addr 172.21.15.98:0/3592259835 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:23.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.040+0000 7f61361f7640 1 -- 172.21.15.98:0/3592259835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61301f7650 msgr2=0x7f613016c220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T20:58:23.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.040+0000 7f61361f7640 1 --2- 172.21.15.98:0/3592259835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61301f7650 0x7f613016c220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.040+0000 7f61361f7640 1 -- 172.21.15.98:0/3592259835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61301f7270 msgr2=0x7f61301f4c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T20:58:23.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.040+0000 7f61361f7640 1 --2- 172.21.15.98:0/3592259835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61301f7270 0x7f61301f4c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.040+0000 7f61361f7640 1 -- 172.21.15.98:0/3592259835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61301f4190 con 0x7f613016c760 2024-09-10T20:58:23.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.040+0000 7f61361f7640 1 --2- 172.21.15.98:0/3592259835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f613016c760 0x7f61302016b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f612002a8f0 tx=0x7f61200305e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:23.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.041+0000 7fe97b565640 1 --2- 172.21.15.98:0/630055092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9740b2cd0 0x7fe9740b4960 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe974070be0 tx=0x7fe96c02a440 comp rx=0 tx=0).stop 2024-09-10T20:58:23.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.041+0000 7f61267fc640 1 -- 172.21.15.98:0/3592259835 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120040030 con 0x7f613016c760 2024-09-10T20:58:23.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.043+0000 7fe97b565640 1 -- 172.21.15.98:0/630055092 shutdown_connections 2024-09-10T20:58:23.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.043+0000 7fe97b565640 1 --2- 172.21.15.98:0/630055092 >> v2:172.21.15.190:3300/0 conn(0x7fe9740b30b0 0x7fe974161f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.041+0000 7f6137c5a640 1 -- 172.21.15.98:0/3592259835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61302023b0 con 0x7f613016c760 2024-09-10T20:58:23.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.041+0000 7f6137c5a640 1 -- 172.21.15.98:0/3592259835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61301f4730 con 0x7f613016c760 2024-09-10T20:58:23.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.043+0000 7f61267fc640 1 -- 172.21.15.98:0/3592259835 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6120005110 con 0x7f613016c760 2024-09-10T20:58:23.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.043+0000 7fe97b565640 1 --2- 172.21.15.98:0/630055092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe974072560 0x7fe974161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.043+0000 7fe97b565640 1 --2- 172.21.15.98:0/630055092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9740b2cd0 0x7fe9740b4960 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.043+0000 7fe97b565640 1 -- 172.21.15.98:0/630055092 >> 172.21.15.98:0/630055092 conn(0x7fe9740cfaf0 msgr2=0x7fe9740cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.043+0000 7f61267fc640 1 -- 172.21.15.98:0/3592259835 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120037040 con 0x7f613016c760 2024-09-10T20:58:23.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.043+0000 7f61267fc640 1 -- 172.21.15.98:0/3592259835 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f612004d060 con 0x7f613016c760 2024-09-10T20:58:23.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.043+0000 7f61267fc640 1 -- 172.21.15.98:0/3592259835 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6120004ae0 con 0x7f613016c760 2024-09-10T20:58:23.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.045+0000 7f79716e6640 1 -- 172.21.15.98:0/1493652241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79640deee0 msgr2=0x7f7964036010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.045+0000 7f79716e6640 1 --2- 172.21.15.98:0/1493652241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79640deee0 0x7f7964036010 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7964033a50 tx=0x7f796002e840 comp rx=0 tx=0).stop 2024-09-10T20:58:23.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.046+0000 7fb8e10b6640 1 -- 172.21.15.98:0/988092118 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fb8bc005580 con 0x7fb8dc16e1f0 2024-09-10T20:58:23.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.048+0000 7f99d1ffb640 1 -- 172.21.15.98:0/4062530651 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f99cc005580 con 0x7f99ec0ad890 2024-09-10T20:58:23.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.049+0000 7f6137c5a640 1 -- 172.21.15.98:0/3592259835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6118005290 con 0x7f613016c760 2024-09-10T20:58:23.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.049+0000 7fc267c01640 1 -- 172.21.15.98:0/2201353592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc26015b1e0 msgr2=0x7fc26006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.049+0000 7fc267c01640 1 --2- 172.21.15.98:0/2201353592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc26015b1e0 0x7fc26006fc90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc26006c960 tx=0x7fc25c02a3d0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.048+0000 7f79716e6640 1 -- 172.21.15.98:0/1493652241 shutdown_connections 2024-09-10T20:58:23.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.048+0000 7f79716e6640 1 --2- 172.21.15.98:0/1493652241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79640deee0 0x7f7964036010 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.048+0000 7f79716e6640 1 -- 172.21.15.98:0/1493652241 >> 172.21.15.98:0/1493652241 conn(0x7f796401e000 msgr2=0x7f7964054df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.051+0000 7fc267c01640 1 -- 172.21.15.98:0/2201353592 shutdown_connections 2024-09-10T20:58:23.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.051+0000 7fc267c01640 1 --2- 172.21.15.98:0/2201353592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc26015b1e0 0x7fc26006fc90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.051+0000 7fc267c01640 1 --2- 172.21.15.98:0/2201353592 >> v2:172.21.15.190:3300/0 conn(0x7fc26013cf80 0x7fc26006f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.051+0000 7fc267c01640 1 -- 172.21.15.98:0/2201353592 >> 172.21.15.98:0/2201353592 conn(0x7fc2600b1760 msgr2=0x7fc2600b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.051+0000 7f79716e6640 1 -- 172.21.15.98:0/1493652241 shutdown_connections 2024-09-10T20:58:23.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.051+0000 7f79716e6640 1 -- 172.21.15.98:0/1493652241 wait complete. 2024-09-10T20:58:23.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.053+0000 7f79716e6640 1 Processor -- start 2024-09-10T20:58:23.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.053+0000 7f79716e6640 1 -- start start 2024-09-10T20:58:23.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.054+0000 7f79716e6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79640345b0 0x7f79640d6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7f796affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79640345b0 0x7f79640d6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7f79716e6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f796403a250 0x7f7964189800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7f79716e6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79640deee0 0x7f7964182e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7f796affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79640345b0 0x7f79640d6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49752/0 (socket says 172.21.15.98:49752) 2024-09-10T20:58:23.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7f796affd640 1 -- 172.21.15.98:0/3213666066 learned_addr learned my addr 172.21.15.98:0/3213666066 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:23.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7f79716e6640 1 -- 172.21.15.98:0/3213666066 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7964006120 con 0x7f79640345b0 2024-09-10T20:58:23.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7f79716e6640 1 -- 172.21.15.98:0/3213666066 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7964005fa0 con 0x7f79640deee0 2024-09-10T20:58:23.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7f79716e6640 1 -- 172.21.15.98:0/3213666066 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f79640062a0 con 0x7f796403a250 2024-09-10T20:58:23.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7f796b7fe640 1 --2- 172.21.15.98:0/3213666066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79640deee0 0x7f7964182e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7f796a7fc640 1 --2- 172.21.15.98:0/3213666066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f796403a250 0x7f7964189800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.056+0000 7f796affd640 1 -- 172.21.15.98:0/3213666066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f796403a250 msgr2=0x7f7964189800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.056+0000 7f796affd640 1 --2- 172.21.15.98:0/3213666066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f796403a250 0x7f7964189800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.056+0000 7f796affd640 1 -- 172.21.15.98:0/3213666066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79640deee0 msgr2=0x7f7964182e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.056+0000 7f796affd640 1 --2- 172.21.15.98:0/3213666066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79640deee0 0x7f7964182e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.056+0000 7f796affd640 1 -- 172.21.15.98:0/3213666066 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79640d9340 con 0x7f79640345b0 2024-09-10T20:58:23.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.056+0000 7f796affd640 1 --2- 172.21.15.98:0/3213666066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79640345b0 0x7f79640d6020 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f796000d9d0 tx=0x7f79600056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:23.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.057+0000 7f79537fe640 1 -- 172.21.15.98:0/3213666066 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960044070 con 0x7f79640345b0 2024-09-10T20:58:23.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.057+0000 7f79537fe640 1 -- 172.21.15.98:0/3213666066 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7960004070 con 0x7f79640345b0 2024-09-10T20:58:23.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.057+0000 7f79537fe640 1 -- 172.21.15.98:0/3213666066 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796003a030 con 0x7f79640345b0 2024-09-10T20:58:23.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.058+0000 7f79716e6640 1 -- 172.21.15.98:0/3213666066 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79640d6560 con 0x7f79640345b0 2024-09-10T20:58:23.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.054+0000 7fb8c37fe640 1 -- 172.21.15.98:0/988092118 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v8)=0 v8) ==== 106+0+0 (secure 0 0 0) 0x7fb8d400ba30 con 0x7fb8dc16e1f0 2024-09-10T20:58:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7fb8c17fa640 1 -- 172.21.15.98:0/988092118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8dc16e1f0 msgr2=0x7fb8dc166c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7fb8c17fa640 1 --2- 172.21.15.98:0/988092118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8dc16e1f0 0x7fb8dc166c00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400efc0 tx=0x7fb8d400c800 comp rx=0 tx=0).stop 2024-09-10T20:58:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.058+0000 7fb8c17fa640 1 -- 172.21.15.98:0/988092118 shutdown_connections 2024-09-10T20:58:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.058+0000 7fb8c17fa640 1 --2- 172.21.15.98:0/988092118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8dc167140 0x7fb8dc16ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.058+0000 7fb8c17fa640 1 --2- 172.21.15.98:0/988092118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8dc16e1f0 0x7fb8dc166c00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.058+0000 7fb8c17fa640 1 --2- 172.21.15.98:0/988092118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8dc0cf400 0x7fb8dc155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.058+0000 7fb8c17fa640 1 -- 172.21.15.98:0/988092118 >> 172.21.15.98:0/988092118 conn(0x7fb8dc0af660 msgr2=0x7fb8dc0b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7fc267c01640 1 -- 172.21.15.98:0/2201353592 shutdown_connections 2024-09-10T20:58:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7fc267c01640 1 -- 172.21.15.98:0/2201353592 wait complete. 2024-09-10T20:58:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.057+0000 7fc267c01640 1 Processor -- start 2024-09-10T20:58:23.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.055+0000 7f61267fc640 1 -- 172.21.15.98:0/3592259835 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f6120005560 con 0x7f613016c760 2024-09-10T20:58:23.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.051+0000 7fe97b565640 1 -- 172.21.15.98:0/630055092 shutdown_connections 2024-09-10T20:58:23.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.051+0000 7fe97b565640 1 -- 172.21.15.98:0/630055092 wait complete. 2024-09-10T20:58:23.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.053+0000 7fe97b565640 1 Processor -- start 2024-09-10T20:58:23.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.053+0000 7fe97b565640 1 -- start start 2024-09-10T20:58:23.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.054+0000 7fe97b565640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe974072560 0x7fe974141020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.054+0000 7fe97b565640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9740b2cd0 0x7fe97413a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.054+0000 7fe97b565640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9740b30b0 0x7fe974146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.054+0000 7fe97b565640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe974142c60 con 0x7fe974072560 2024-09-10T20:58:23.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.058+0000 7f79716e6640 1 -- 172.21.15.98:0/3213666066 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79640d8d80 con 0x7f79640345b0 2024-09-10T20:58:23.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.057+0000 7fc267c01640 1 -- start start 2024-09-10T20:58:23.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.054+0000 7fe97b565640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe9741627c0 con 0x7fe9740b30b0 2024-09-10T20:58:23.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.054+0000 7fe97b565640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe974142de0 con 0x7fe9740b2cd0 2024-09-10T20:58:23.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fe979b02640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9740b30b0 0x7fe974146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.058+0000 7fc267c01640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc26013cf80 0x7fc260150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fc267c01640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc26015b1e0 0x7fc2600d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fe979b02640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9740b30b0 0x7fe974146350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:60754/0 (socket says 172.21.15.98:60754) 2024-09-10T20:58:23.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fe979b02640 1 -- 172.21.15.98:0/3225615037 learned_addr learned my addr 172.21.15.98:0/3225615037 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:23.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fe978b00640 1 --2- 172.21.15.98:0/3225615037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9740b2cd0 0x7fe97413a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fc267c01640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26015da30 0x7fc2600cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fc26519c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc26015b1e0 0x7fc2600d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fc26519c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc26015b1e0 0x7fc2600d04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:45118/0 (socket says 172.21.15.98:45118) 2024-09-10T20:58:23.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fc26519c640 1 -- 172.21.15.98:0/3467588636 learned_addr learned my addr 172.21.15.98:0/3467588636 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:23.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fc267c01640 1 -- 172.21.15.98:0/3467588636 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc2600b3e90 con 0x7fc26013cf80 2024-09-10T20:58:23.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fc267c01640 1 -- 172.21.15.98:0/3467588636 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc2600b3d10 con 0x7fc26015da30 2024-09-10T20:58:23.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fc267c01640 1 -- 172.21.15.98:0/3467588636 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc2600b4010 con 0x7fc26015b1e0 2024-09-10T20:58:23.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fc26619e640 1 --2- 172.21.15.98:0/3467588636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26015da30 0x7fc2600cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fc26619e640 1 -- 172.21.15.98:0/3467588636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc26015b1e0 msgr2=0x7fc2600d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fc26619e640 1 --2- 172.21.15.98:0/3467588636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc26015b1e0 0x7fc2600d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fc26619e640 1 -- 172.21.15.98:0/3467588636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc26013cf80 msgr2=0x7fc260150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fc26619e640 1 --2- 172.21.15.98:0/3467588636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc26013cf80 0x7fc260150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fc26619e640 1 -- 172.21.15.98:0/3467588636 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2600cf430 con 0x7fc26015da30 2024-09-10T20:58:23.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fc26619e640 1 --2- 172.21.15.98:0/3467588636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26015da30 0x7fc2600cea40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc25800c7a0 tx=0x7fc25800cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:23.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fc9b7079640 1 -- 172.21.15.98:0/390091272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc99801b4e0 msgr2=0x7fc99801d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fc9b7079640 1 --2- 172.21.15.98:0/390091272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc99801b4e0 0x7fc99801d970 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4006ef0 tx=0x7fc998005830 comp rx=0 tx=0).stop 2024-09-10T20:58:23.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fc9b5616640 1 -- 172.21.15.98:0/390091272 reap_dead start 2024-09-10T20:58:23.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fc9b7079640 1 -- 172.21.15.98:0/390091272 shutdown_connections 2024-09-10T20:58:23.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fc9b7079640 1 -- 172.21.15.98:0/390091272 >> 172.21.15.98:0/390091272 conn(0x7fc9b006e480 msgr2=0x7fc9b00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fc9b7079640 1 -- 172.21.15.98:0/390091272 shutdown_connections 2024-09-10T20:58:23.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fc9b7079640 1 -- 172.21.15.98:0/390091272 wait complete. 2024-09-10T20:58:23.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fe979b02640 1 -- 172.21.15.98:0/3225615037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9740b2cd0 msgr2=0x7fe97413a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fe979b02640 1 --2- 172.21.15.98:0/3225615037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9740b2cd0 0x7fe97413a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fe979b02640 1 -- 172.21.15.98:0/3225615037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe974072560 msgr2=0x7fe974141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fe979b02640 1 --2- 172.21.15.98:0/3225615037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe974072560 0x7fe974141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.059+0000 7fe979b02640 1 -- 172.21.15.98:0/3225615037 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9742029b0 con 0x7fe9740b30b0 2024-09-10T20:58:23.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fe979b02640 1 --2- 172.21.15.98:0/3225615037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9740b30b0 0x7fe974146350 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe96400e660 tx=0x7fe96400eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:23.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.060+0000 7fe969ffb640 1 -- 172.21.15.98:0/3225615037 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96400cab0 con 0x7fe9740b30b0 2024-09-10T20:58:23.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.061+0000 7fe969ffb640 1 -- 172.21.15.98:0/3225615037 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe96400cc70 con 0x7fe9740b30b0 2024-09-10T20:58:23.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.061+0000 7fe969ffb640 1 -- 172.21.15.98:0/3225615037 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96400ce60 con 0x7fe9740b30b0 2024-09-10T20:58:23.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.061+0000 7fc9b7079640 1 Processor -- start 2024-09-10T20:58:23.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.062+0000 7fe97b565640 1 -- 172.21.15.98:0/3225615037 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe974201e30 con 0x7fe9740b30b0 2024-09-10T20:58:23.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.062+0000 7fe97b565640 1 -- 172.21.15.98:0/3225615037 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9742023f0 con 0x7fe9740b30b0 2024-09-10T20:58:23.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.062+0000 7f79537fe640 1 -- 172.21.15.98:0/3213666066 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f7960035070 con 0x7f79640345b0 2024-09-10T20:58:23.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.062+0000 7f79537fe640 1 -- 172.21.15.98:0/3213666066 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7960034500 con 0x7f79640345b0 2024-09-10T20:58:23.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.062+0000 7fc2527fc640 1 -- 172.21.15.98:0/3467588636 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258017070 con 0x7fc26015da30 2024-09-10T20:58:23.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.062+0000 7fc2527fc640 1 -- 172.21.15.98:0/3467588636 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc258005070 con 0x7fc26015da30 2024-09-10T20:58:23.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.062+0000 7fc2527fc640 1 -- 172.21.15.98:0/3467588636 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2580054c0 con 0x7fc26015da30 2024-09-10T20:58:23.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.063+0000 7fc267c01640 1 -- 172.21.15.98:0/3467588636 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2600cf710 con 0x7fc26015da30 2024-09-10T20:58:23.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.063+0000 7fc267c01640 1 -- 172.21.15.98:0/3467588636 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2600cfcd0 con 0x7fc26015da30 2024-09-10T20:58:23.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.065+0000 7fb8c17fa640 1 -- 172.21.15.98:0/988092118 shutdown_connections 2024-09-10T20:58:23.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.065+0000 7f99d3fff640 1 -- 172.21.15.98:0/4062530651 <== mon.1 v2:172.21.15.190:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v9)=0 v9) ==== 117+0+0 (secure 0 0 0) 0x7f99e0019560 con 0x7f99ec0ad890 2024-09-10T20:58:23.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.065+0000 7fb8c17fa640 1 -- 172.21.15.98:0/988092118 wait complete. 2024-09-10T20:58:23.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.065+0000 7fe97b565640 1 -- 172.21.15.98:0/3225615037 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9740b3f80 con 0x7fe9740b30b0 2024-09-10T20:58:23.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.066+0000 7fe969ffb640 1 -- 172.21.15.98:0/3225615037 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fe9640047a0 con 0x7fe9740b30b0 2024-09-10T20:58:23.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.066+0000 7fe969ffb640 1 -- 172.21.15.98:0/3225615037 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe964021df0 con 0x7fe9740b30b0 2024-09-10T20:58:23.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.067+0000 7fc9b7079640 1 -- start start 2024-09-10T20:58:23.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.068+0000 7f79716e6640 1 -- 172.21.15.98:0/3213666066 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7964036010 con 0x7f79640345b0 2024-09-10T20:58:23.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.072+0000 7f79537fe640 1 -- 172.21.15.98:0/3213666066 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f796004f020 con 0x7f79640345b0 2024-09-10T20:58:23.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.071+0000 7f99d1ffb640 1 -- 172.21.15.98:0/4062530651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99ec0ad890 msgr2=0x7f99ec1f9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.071+0000 7f99d1ffb640 1 --2- 172.21.15.98:0/4062530651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99ec0ad890 0x7f99ec1f9410 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f99e000b3e0 tx=0x7f99e000b8a0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.072+0000 7f99d1ffb640 1 -- 172.21.15.98:0/4062530651 shutdown_connections 2024-09-10T20:58:23.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.072+0000 7f99d1ffb640 1 --2- 172.21.15.98:0/4062530651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99ec1477b0 0x7f99ec0aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.068+0000 7fc9b7079640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc99801b4e0 0x7fc9b00a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.068+0000 7fc9b7079640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9b0157350 0x7fc9b0156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.068+0000 7fc9b7079640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b00aabe0 0x7fc9b0159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.068+0000 7fc9b7079640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b0161a60 con 0x7fc99801b4e0 2024-09-10T20:58:23.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.068+0000 7fc9b7079640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b015f380 con 0x7fc9b00aabe0 2024-09-10T20:58:23.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.068+0000 7fc9b7079640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b0161be0 con 0x7fc9b0157350 2024-09-10T20:58:23.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.068+0000 7fe969ffb640 1 -- 172.21.15.98:0/3225615037 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fe9740b3f80 con 0x7fe9740b30b0 2024-09-10T20:58:23.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.072+0000 7f99d1ffb640 1 --2- 172.21.15.98:0/4062530651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99ec0b2fa0 0x7f99ec15d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.072+0000 7f99d1ffb640 1 --2- 172.21.15.98:0/4062530651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99ec0ad890 0x7f99ec1f9410 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.072+0000 7f99d1ffb640 1 -- 172.21.15.98:0/4062530651 >> 172.21.15.98:0/4062530651 conn(0x7f99ec0cfaf0 msgr2=0x7f99ec0cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.072+0000 7fc267c01640 1 -- 172.21.15.98:0/3467588636 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc26015de90 con 0x7fc26015da30 2024-09-10T20:58:23.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.070+0000 7fc9b5616640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b00aabe0 0x7fc9b0159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.070+0000 7fc9b5616640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b00aabe0 0x7fc9b0159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:60776/0 (socket says 172.21.15.98:60776) 2024-09-10T20:58:23.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.070+0000 7fc9b5616640 1 -- 172.21.15.98:0/3766299131 learned_addr learned my addr 172.21.15.98:0/3766299131 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:23.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.070+0000 7fc9affff640 1 --2- 172.21.15.98:0/3766299131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9b0157350 0x7fc9b0156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.071+0000 7fc9b5616640 1 -- 172.21.15.98:0/3766299131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9b0157350 msgr2=0x7fc9b0156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.071+0000 7fc9b5616640 1 --2- 172.21.15.98:0/3766299131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9b0157350 0x7fc9b0156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.071+0000 7fc9b5616640 1 -- 172.21.15.98:0/3766299131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc99801b4e0 msgr2=0x7fc9b00a9480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T20:58:23.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.072+0000 7f99d1ffb640 1 -- 172.21.15.98:0/4062530651 shutdown_connections 2024-09-10T20:58:23.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.072+0000 7fc2527fc640 1 -- 172.21.15.98:0/3467588636 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fc258005660 con 0x7fc26015da30 2024-09-10T20:58:23.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.072+0000 7fc2527fc640 1 -- 172.21.15.98:0/3467588636 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc25801c9d0 con 0x7fc26015da30 2024-09-10T20:58:23.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.072+0000 7f99d1ffb640 1 -- 172.21.15.98:0/4062530651 wait complete. 2024-09-10T20:58:23.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.074+0000 7fc2527fc640 1 -- 172.21.15.98:0/3467588636 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fc258021070 con 0x7fc26015da30 2024-09-10T20:58:23.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.077+0000 7fc9b4e15640 1 --2- 172.21.15.98:0/3766299131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc99801b4e0 0x7fc9b00a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.077+0000 7fc9b5616640 1 --2- 172.21.15.98:0/3766299131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc99801b4e0 0x7fc9b00a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.077+0000 7fc9b5616640 1 -- 172.21.15.98:0/3766299131 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b0157bf0 con 0x7fc9b00aabe0 2024-09-10T20:58:23.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.077+0000 7fc9b4e15640 1 --2- 172.21.15.98:0/3766299131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc99801b4e0 0x7fc9b00a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:58:23.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.082+0000 7fc9b5616640 1 --2- 172.21.15.98:0/3766299131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b00aabe0 0x7fc9b0159d60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4053040 tx=0x7fc9a40060d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:23.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.082+0000 7fc9ad7fa640 1 -- 172.21.15.98:0/3766299131 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4054030 con 0x7fc9b00aabe0 2024-09-10T20:58:23.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.082+0000 7fc9ad7fa640 1 -- 172.21.15.98:0/3766299131 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9a40038e0 con 0x7fc9b00aabe0 2024-09-10T20:58:23.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.082+0000 7fc9ad7fa640 1 -- 172.21.15.98:0/3766299131 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4003d10 con 0x7fc9b00aabe0 2024-09-10T20:58:23.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.082+0000 7fc9b7079640 1 -- 172.21.15.98:0/3766299131 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b00a9b40 con 0x7fc9b00aabe0 2024-09-10T20:58:23.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.082+0000 7fc9b7079640 1 -- 172.21.15.98:0/3766299131 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b00aa100 con 0x7fc9b00aabe0 2024-09-10T20:58:23.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.083+0000 7fc9ad7fa640 1 -- 172.21.15.98:0/3766299131 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fc9a40597b0 con 0x7fc9b00aabe0 2024-09-10T20:58:23.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.083+0000 7fc9ad7fa640 1 -- 172.21.15.98:0/3766299131 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc9a40507d0 con 0x7fc9b00aabe0 2024-09-10T20:58:23.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.085+0000 7fc9b7079640 1 -- 172.21.15.98:0/3766299131 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc990005290 con 0x7fc9b00aabe0 2024-09-10T20:58:23.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.090+0000 7fc9ad7fa640 1 -- 172.21.15.98:0/3766299131 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fc9a4069020 con 0x7fc9b00aabe0 2024-09-10T20:58:23.097 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi098') 2024-09-10T20:58:23.097 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi098') 2024-09-10T20:58:23.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.109+0000 7f643a2f7640 1 -- 172.21.15.98:0/1162716562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6434072560 msgr2=0x7f64340b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.109+0000 7f643a2f7640 1 --2- 172.21.15.98:0/1162716562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6434072560 0x7f64340b4960 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6428005fd0 tx=0x7f642802a560 comp rx=0 tx=0).stop 2024-09-10T20:58:23.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.114+0000 7f643a2f7640 1 -- 172.21.15.98:0/1162716562 shutdown_connections 2024-09-10T20:58:23.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.114+0000 7f643a2f7640 1 --2- 172.21.15.98:0/1162716562 >> v2:172.21.15.190:3300/0 conn(0x7f64340b30b0 0x7f6434162130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.114+0000 7f643a2f7640 1 --2- 172.21.15.98:0/1162716562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6434072560 0x7f64340b4960 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.114+0000 7f643a2f7640 1 -- 172.21.15.98:0/1162716562 >> 172.21.15.98:0/1162716562 conn(0x7f64340cfaf0 msgr2=0x7f64340cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.120+0000 7f643a2f7640 1 -- 172.21.15.98:0/1162716562 shutdown_connections 2024-09-10T20:58:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.125+0000 7f2a8a89b640 1 -- 172.21.15.98:0/2160386639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a8406ccc0 msgr2=0x7f2a84161a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.125+0000 7f2a8a89b640 1 --2- 172.21.15.98:0/2160386639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a8406ccc0 0x7f2a84161a60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2a840ce6e0 tx=0x7f2a7402a460 comp rx=0 tx=0).stop 2024-09-10T20:58:23.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.128+0000 7f2a8a89b640 1 -- 172.21.15.98:0/2160386639 shutdown_connections 2024-09-10T20:58:23.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.128+0000 7f2a8a89b640 1 --2- 172.21.15.98:0/2160386639 >> v2:172.21.15.98:3300/0 conn(0x7f2a840b5cf0 0x7f2a8415ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.128+0000 7f2a8a89b640 1 --2- 172.21.15.98:0/2160386639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a8406ccc0 0x7f2a84161a60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.128+0000 7f2a8a89b640 1 -- 172.21.15.98:0/2160386639 >> 172.21.15.98:0/2160386639 conn(0x7f2a8406e480 msgr2=0x7f2a840b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.131+0000 7f643a2f7640 1 -- 172.21.15.98:0/1162716562 wait complete. 2024-09-10T20:58:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.132+0000 7f643a2f7640 1 Processor -- start 2024-09-10T20:58:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.133+0000 7f643a2f7640 1 -- start start 2024-09-10T20:58:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.134+0000 7f643a2f7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6434072560 0x7f6434156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.134+0000 7f643a2f7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64340b2cd0 0x7f643408b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.134+0000 7f643a2f7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64340b30b0 0x7f643408b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.134+0000 7f643a2f7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f64340cdc00 con 0x7f6434072560 2024-09-10T20:58:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.134+0000 7f643a2f7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f64340cda80 con 0x7f64340b2cd0 2024-09-10T20:58:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.134+0000 7f643a2f7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f64340cdd80 con 0x7f64340b30b0 2024-09-10T20:58:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.134+0000 7f6433fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6434072560 0x7f6434156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.134+0000 7f6433fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6434072560 0x7f6434156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49794/0 (socket says 172.21.15.98:49794) 2024-09-10T20:58:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.134+0000 7f6433fff640 1 -- 172.21.15.98:0/2796974624 learned_addr learned my addr 172.21.15.98:0/2796974624 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:23.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.135+0000 7fd3fea03640 1 -- 172.21.15.98:0/236264388 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fd3d8005580 con 0x7fd3f80cf400 2024-09-10T20:58:23.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.135+0000 7f6433fff640 1 -- 172.21.15.98:0/2796974624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64340b30b0 msgr2=0x7f643408b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T20:58:23.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.135+0000 7f6433fff640 1 --2- 172.21.15.98:0/2796974624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64340b30b0 0x7f643408b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.135+0000 7f6433fff640 1 -- 172.21.15.98:0/2796974624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64340b2cd0 msgr2=0x7f643408b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.135+0000 7f6433fff640 1 --2- 172.21.15.98:0/2796974624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64340b2cd0 0x7f643408b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.135+0000 7f6433fff640 1 -- 172.21.15.98:0/2796974624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64341567a0 con 0x7f6434072560 2024-09-10T20:58:23.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.136+0000 7f6433fff640 1 --2- 172.21.15.98:0/2796974624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6434072560 0x7f6434156260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f642400c4c0 tx=0x7f642400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:23.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.136+0000 7f6430ff9640 1 -- 172.21.15.98:0/2796974624 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64240052d0 con 0x7f6434072560 2024-09-10T20:58:23.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.137+0000 7f643a2f7640 1 -- 172.21.15.98:0/2796974624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643413fcf0 con 0x7f6434072560 2024-09-10T20:58:23.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.137+0000 7f643a2f7640 1 -- 172.21.15.98:0/2796974624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643406e410 con 0x7f6434072560 2024-09-10T20:58:23.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.138+0000 7f6430ff9640 1 -- 172.21.15.98:0/2796974624 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6424005490 con 0x7f6434072560 2024-09-10T20:58:23.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.139+0000 7fd3f4ff9640 1 -- 172.21.15.98:0/236264388 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7fd3e8020bb0 con 0x7fd3f80cf400 2024-09-10T20:58:23.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.138+0000 7f6430ff9640 1 -- 172.21.15.98:0/2796974624 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424012950 con 0x7f6434072560 2024-09-10T20:58:23.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.140+0000 7f641a7fc640 1 -- 172.21.15.98:0/2796974624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6414005290 con 0x7f6434072560 2024-09-10T20:58:23.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.142+0000 7f6430ff9640 1 -- 172.21.15.98:0/2796974624 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f64240035f0 con 0x7f6434072560 2024-09-10T20:58:23.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.142+0000 7fd3fea03640 1 -- 172.21.15.98:0/236264388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3f80cf400 msgr2=0x7fd3f8089b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.142+0000 7fd3fea03640 1 --2- 172.21.15.98:0/236264388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3f80cf400 0x7fd3f8089b90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800fba0 tx=0x7fd3e800d7b0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.142+0000 7fd3fea03640 1 -- 172.21.15.98:0/236264388 shutdown_connections 2024-09-10T20:58:23.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.142+0000 7fd3fea03640 1 --2- 172.21.15.98:0/236264388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3f80c51c0 0x7fd3f80b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.142+0000 7fd3fea03640 1 --2- 172.21.15.98:0/236264388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f816e1f0 0x7fd3f8169b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.142+0000 7fd3fea03640 1 --2- 172.21.15.98:0/236264388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3f80cf400 0x7fd3f8089b90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.142+0000 7fd3fea03640 1 -- 172.21.15.98:0/236264388 >> 172.21.15.98:0/236264388 conn(0x7fd3f80af660 msgr2=0x7fd3f816e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.142+0000 7fd3fea03640 1 -- 172.21.15.98:0/236264388 shutdown_connections 2024-09-10T20:58:23.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.142+0000 7fd3fea03640 1 -- 172.21.15.98:0/236264388 wait complete. 2024-09-10T20:58:23.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.144+0000 7f2a8a89b640 1 -- 172.21.15.98:0/2160386639 shutdown_connections 2024-09-10T20:58:23.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.144+0000 7f2a8a89b640 1 -- 172.21.15.98:0/2160386639 wait complete. 2024-09-10T20:58:23.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.145+0000 7f2a8a89b640 1 Processor -- start 2024-09-10T20:58:23.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.146+0000 7f2a8a89b640 1 -- start start 2024-09-10T20:58:23.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.147+0000 7f2a8a89b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a8406ccc0 0x7f2a842019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.147+0000 7f2a8a89b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a8406d0a0 0x7f2a842023c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.147+0000 7f2a8a89b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a840b5cf0 0x7f2a84202b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.147+0000 7f2a8a89b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8415f370 con 0x7f2a8406d0a0 2024-09-10T20:58:23.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.147+0000 7f2a8a89b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8415f1f0 con 0x7f2a840b5cf0 2024-09-10T20:58:23.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.147+0000 7f2a8a89b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2a840af4e0 con 0x7f2a8406ccc0 2024-09-10T20:58:23.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.148+0000 7f2a83fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a8406ccc0 0x7f2a842019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.148+0000 7f2a83fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a8406ccc0 0x7f2a842019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:45150/0 (socket says 172.21.15.98:45150) 2024-09-10T20:58:23.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.148+0000 7f2a83fff640 1 -- 172.21.15.98:0/248638651 learned_addr learned my addr 172.21.15.98:0/248638651 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:23.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.148+0000 7f2a88e38640 1 --2- 172.21.15.98:0/248638651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a840b5cf0 0x7f2a84202b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.148+0000 7f2a83fff640 1 -- 172.21.15.98:0/248638651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a840b5cf0 msgr2=0x7f2a84202b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.148+0000 7f2a83fff640 1 --2- 172.21.15.98:0/248638651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a840b5cf0 0x7f2a84202b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.148+0000 7f2a83fff640 1 -- 172.21.15.98:0/248638651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a8406d0a0 msgr2=0x7f2a842023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.148+0000 7f2a83fff640 1 --2- 172.21.15.98:0/248638651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a8406d0a0 0x7f2a842023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.148+0000 7f2a83fff640 1 -- 172.21.15.98:0/248638651 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a84202900 con 0x7f2a8406ccc0 2024-09-10T20:58:23.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.150+0000 7f2a83fff640 1 --2- 172.21.15.98:0/248638651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a8406ccc0 0x7f2a842019a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009920 tx=0x7f2a74030570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:23.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.150+0000 7f6430ff9640 1 -- 172.21.15.98:0/2796974624 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f642401b8e0 con 0x7f6434072560 2024-09-10T20:58:23.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.150+0000 7f6430ff9640 1 -- 172.21.15.98:0/2796974624 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f6424020060 con 0x7f6434072560 2024-09-10T20:58:23.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.151+0000 7f2a80ff9640 1 -- 172.21.15.98:0/248638651 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74040030 con 0x7f2a8406ccc0 2024-09-10T20:58:23.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.151+0000 7f2a8a89b640 1 -- 172.21.15.98:0/248638651 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a842033e0 con 0x7f2a8406ccc0 2024-09-10T20:58:23.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.151+0000 7f2a8a89b640 1 -- 172.21.15.98:0/248638651 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a842039a0 con 0x7f2a8406ccc0 2024-09-10T20:58:23.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.152+0000 7f2a80ff9640 1 -- 172.21.15.98:0/248638651 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a74004110 con 0x7f2a8406ccc0 2024-09-10T20:58:23.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.152+0000 7f2a80ff9640 1 -- 172.21.15.98:0/248638651 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74037040 con 0x7f2a8406ccc0 2024-09-10T20:58:23.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.152+0000 7f410a7fc640 1 -- 172.21.15.98:0/1402821046 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f40f8005580 con 0x7f41140b59e0 2024-09-10T20:58:23.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.153+0000 7f2a80ff9640 1 -- 172.21.15.98:0/248638651 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f2a7404d060 con 0x7f2a8406ccc0 2024-09-10T20:58:23.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.153+0000 7f2a80ff9640 1 -- 172.21.15.98:0/248638651 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2a74003930 con 0x7f2a8406ccc0 2024-09-10T20:58:23.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.153+0000 7f2a8a89b640 1 -- 172.21.15.98:0/248638651 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a70005290 con 0x7f2a8406ccc0 2024-09-10T20:58:23.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.157+0000 7f2a80ff9640 1 -- 172.21.15.98:0/248638651 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f2a74032630 con 0x7f2a8406ccc0 2024-09-10T20:58:23.164 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi098') 2024-09-10T20:58:23.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.160+0000 7f4110ff9640 1 -- 172.21.15.98:0/1402821046 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7f40fc043440 con 0x7f41140b59e0 2024-09-10T20:58:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.166+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/3628122867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a40b2fa0 msgr2=0x7ff1a4072b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.166+0000 7ff1a8a6e640 1 --2- 172.21.15.98:0/3628122867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a40b2fa0 0x7ff1a4072b90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40cfaa0 tx=0x7ff198032b10 comp rx=0 tx=0).stop 2024-09-10T20:58:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.166+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/3628122867 shutdown_connections 2024-09-10T20:58:23.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.166+0000 7ff1a8a6e640 1 --2- 172.21.15.98:0/3628122867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a40b2fa0 0x7ff1a4072b90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.166+0000 7ff1a8a6e640 1 --2- 172.21.15.98:0/3628122867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40ad890 0x7ff1a40b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.166+0000 7ff1a8a6e640 1 --2- 172.21.15.98:0/3628122867 >> v2:172.21.15.98:3300/0 conn(0x7ff1a41477b0 0x7ff1a40b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.166+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/3628122867 >> 172.21.15.98:0/3628122867 conn(0x7ff1a40cfaf0 msgr2=0x7ff1a40cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.166+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/3628122867 shutdown_connections 2024-09-10T20:58:23.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.166+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/3628122867 wait complete. 2024-09-10T20:58:23.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.168+0000 7ff1a8a6e640 1 Processor -- start 2024-09-10T20:58:23.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.169+0000 7ff1a8a6e640 1 -- start start 2024-09-10T20:58:23.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.170+0000 7ff1a8a6e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40ad890 0x7ff1a40ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.170+0000 7ff1a8a6e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a40b2fa0 0x7ff1a40aca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.170+0000 7ff1a259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40ad890 0x7ff1a40ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.170+0000 7ff1a259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40ad890 0x7ff1a40ac4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:45164/0 (socket says 172.21.15.98:45164) 2024-09-10T20:58:23.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7ff1a1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a40b2fa0 0x7ff1a40aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7ff1a1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a40b2fa0 0x7ff1a40aca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49826/0 (socket says 172.21.15.98:49826) 2024-09-10T20:58:23.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7ff1a259c640 1 -- 172.21.15.98:0/1702992180 learned_addr learned my addr 172.21.15.98:0/1702992180 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:23.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7ff1a8a6e640 1 --2- 172.21.15.98:0/1702992180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a41477b0 0x7ff1a40c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7f411a639640 1 -- 172.21.15.98:0/1402821046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41140b59e0 msgr2=0x7f41140c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7f411a639640 1 --2- 172.21.15.98:0/1402821046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41140b59e0 0x7f41140c6360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0309d0 tx=0x7f40fc030e90 comp rx=0 tx=0).stop 2024-09-10T20:58:23.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/1702992180 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a4073460 con 0x7ff1a40b2fa0 2024-09-10T20:58:23.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/1702992180 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a40732e0 con 0x7ff1a41477b0 2024-09-10T20:58:23.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7ff1a2d9d640 1 --2- 172.21.15.98:0/1702992180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a41477b0 0x7ff1a40c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/1702992180 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a40735e0 con 0x7ff1a40ad890 2024-09-10T20:58:23.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7f411a639640 1 -- 172.21.15.98:0/1402821046 shutdown_connections 2024-09-10T20:58:23.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7f411a639640 1 --2- 172.21.15.98:0/1402821046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41140d0a60 0x7f4114204c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7f411a639640 1 --2- 172.21.15.98:0/1402821046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41140b59e0 0x7f41140c6360 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7f411a639640 1 --2- 172.21.15.98:0/1402821046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f411406cc60 0x7f41140c2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.171+0000 7f411a639640 1 -- 172.21.15.98:0/1402821046 >> 172.21.15.98:0/1402821046 conn(0x7f4114074170 msgr2=0x7f41140ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.172+0000 7f411a639640 1 -- 172.21.15.98:0/1402821046 shutdown_connections 2024-09-10T20:58:23.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.172+0000 7f411a639640 1 -- 172.21.15.98:0/1402821046 wait complete. 2024-09-10T20:58:23.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.172+0000 7ff1a1d9b640 1 -- 172.21.15.98:0/1702992180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40ad890 msgr2=0x7ff1a40ac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.172+0000 7ff1a1d9b640 1 --2- 172.21.15.98:0/1702992180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40ad890 0x7ff1a40ac4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.172+0000 7ff1a1d9b640 1 -- 172.21.15.98:0/1702992180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a41477b0 msgr2=0x7ff1a40c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.172+0000 7ff1a1d9b640 1 --2- 172.21.15.98:0/1702992180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a41477b0 0x7ff1a40c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.172+0000 7ff1a1d9b640 1 -- 172.21.15.98:0/1702992180 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a4138a30 con 0x7ff1a40b2fa0 2024-09-10T20:58:23.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.173+0000 7ff1a1d9b640 1 --2- 172.21.15.98:0/1702992180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a40b2fa0 0x7ff1a40aca20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff19400b750 tx=0x7ff19400bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:23.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.173+0000 7ff192ffd640 1 -- 172.21.15.98:0/1702992180 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194019070 con 0x7ff1a40b2fa0 2024-09-10T20:58:23.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.173+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/1702992180 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a4138d10 con 0x7ff1a40b2fa0 2024-09-10T20:58:23.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.173+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/1702992180 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a41392d0 con 0x7ff1a40b2fa0 2024-09-10T20:58:23.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.174+0000 7ff192ffd640 1 -- 172.21.15.98:0/1702992180 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff19400c830 con 0x7ff1a40b2fa0 2024-09-10T20:58:23.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.174+0000 7ff192ffd640 1 -- 172.21.15.98:0/1702992180 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19400cc80 con 0x7ff1a40b2fa0 2024-09-10T20:58:23.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.175+0000 7ff192ffd640 1 -- 172.21.15.98:0/1702992180 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7ff19401e440 con 0x7ff1a40b2fa0 2024-09-10T20:58:23.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.175+0000 7ff192ffd640 1 -- 172.21.15.98:0/1702992180 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff194014c50 con 0x7ff1a40b2fa0 2024-09-10T20:58:23.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.179+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/1702992180 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff188005290 con 0x7ff1a40b2fa0 2024-09-10T20:58:23.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.187+0000 7ff192ffd640 1 -- 172.21.15.98:0/1702992180 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7ff194023020 con 0x7ff1a40b2fa0 2024-09-10T20:58:23.194 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi098') 2024-09-10T20:58:23.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.212+0000 7f79716e6640 1 -- 172.21.15.98:0/3213666066 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f79640091a0 con 0x7f79640345b0 2024-09-10T20:58:23.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.215+0000 7f79537fe640 1 -- 172.21.15.98:0/3213666066 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v12)=0 v12) ==== 120+0+0 (secure 0 0 0) 0x7f7960033070 con 0x7f79640345b0 2024-09-10T20:58:23.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.225+0000 7fe97b565640 1 -- 172.21.15.98:0/3225615037 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fe97413b0a0 con 0x7fe9740b30b0 2024-09-10T20:58:23.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.226+0000 7fc9b7079640 1 -- 172.21.15.98:0/3766299131 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fc990005580 con 0x7fc9b00aabe0 2024-09-10T20:58:23.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.227+0000 7f79716e6640 1 -- 172.21.15.98:0/3213666066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79640345b0 msgr2=0x7f79640d6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.227+0000 7f79716e6640 1 --2- 172.21.15.98:0/3213666066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79640345b0 0x7f79640d6020 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f796000d9d0 tx=0x7f79600056f0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.228+0000 7f79716e6640 1 -- 172.21.15.98:0/3213666066 shutdown_connections 2024-09-10T20:58:23.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.228+0000 7f79716e6640 1 --2- 172.21.15.98:0/3213666066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79640deee0 0x7f7964182e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.228+0000 7f79716e6640 1 --2- 172.21.15.98:0/3213666066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f796403a250 0x7f7964189800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.228+0000 7f79716e6640 1 --2- 172.21.15.98:0/3213666066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79640345b0 0x7f79640d6020 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.228+0000 7f79716e6640 1 -- 172.21.15.98:0/3213666066 >> 172.21.15.98:0/3213666066 conn(0x7f796401e000 msgr2=0x7f7964052000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.228+0000 7f79716e6640 1 -- 172.21.15.98:0/3213666066 shutdown_connections 2024-09-10T20:58:23.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.228+0000 7f79716e6640 1 -- 172.21.15.98:0/3213666066 wait complete. 2024-09-10T20:58:23.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.229+0000 7fc267c01640 1 -- 172.21.15.98:0/3467588636 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fc2600cd660 con 0x7fc26015da30 2024-09-10T20:58:23.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.233+0000 7fe969ffb640 1 -- 172.21.15.98:0/3225615037 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v13)=0 v13) ==== 121+0+0 (secure 0 0 0) 0x7fe964026020 con 0x7fe9740b30b0 2024-09-10T20:58:23.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.234+0000 7fe97b565640 1 -- 172.21.15.98:0/3225615037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9740b30b0 msgr2=0x7fe974146350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.234+0000 7fe97b565640 1 --2- 172.21.15.98:0/3225615037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9740b30b0 0x7fe974146350 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe96400e660 tx=0x7fe96400eb20 comp rx=0 tx=0).stop 2024-09-10T20:58:23.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.234+0000 7fe97b565640 1 -- 172.21.15.98:0/3225615037 shutdown_connections 2024-09-10T20:58:23.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.234+0000 7fe97b565640 1 --2- 172.21.15.98:0/3225615037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9740b30b0 0x7fe974146350 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.234+0000 7fe97b565640 1 --2- 172.21.15.98:0/3225615037 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9740b2cd0 0x7fe97413a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.234+0000 7fe97b565640 1 --2- 172.21.15.98:0/3225615037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe974072560 0x7fe974141020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.234+0000 7fe97b565640 1 -- 172.21.15.98:0/3225615037 >> 172.21.15.98:0/3225615037 conn(0x7fe9740cfaf0 msgr2=0x7fe9740cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.235+0000 7fe97b565640 1 -- 172.21.15.98:0/3225615037 shutdown_connections 2024-09-10T20:58:23.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.235+0000 7fe97b565640 1 -- 172.21.15.98:0/3225615037 wait complete. 2024-09-10T20:58:23.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.245+0000 7fc9ad7fa640 1 -- 172.21.15.98:0/3766299131 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v14)=0 v14) ==== 120+0+0 (secure 0 0 0) 0x7fc9b0157bf0 con 0x7fc9b00aabe0 2024-09-10T20:58:23.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.246+0000 7fc996ffd640 1 -- 172.21.15.98:0/3766299131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b00aabe0 msgr2=0x7fc9b0159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.246+0000 7fc996ffd640 1 --2- 172.21.15.98:0/3766299131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b00aabe0 0x7fc9b0159d60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4053040 tx=0x7fc9a40060d0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.247+0000 7fc996ffd640 1 -- 172.21.15.98:0/3766299131 shutdown_connections 2024-09-10T20:58:23.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.247+0000 7fc996ffd640 1 --2- 172.21.15.98:0/3766299131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9b00aabe0 0x7fc9b0159d60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.247+0000 7fc996ffd640 1 --2- 172.21.15.98:0/3766299131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9b0157350 0x7fc9b0156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.247+0000 7fc996ffd640 1 --2- 172.21.15.98:0/3766299131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc99801b4e0 0x7fc9b00a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.247+0000 7fc996ffd640 1 -- 172.21.15.98:0/3766299131 >> 172.21.15.98:0/3766299131 conn(0x7fc9b006e480 msgr2=0x7fc9b0070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.247+0000 7fc996ffd640 1 -- 172.21.15.98:0/3766299131 shutdown_connections 2024-09-10T20:58:23.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.248+0000 7fc996ffd640 1 -- 172.21.15.98:0/3766299131 wait complete. 2024-09-10T20:58:23.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.251+0000 7fc2527fc640 1 -- 172.21.15.98:0/3467588636 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v15)=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7fc25800a5e0 con 0x7fc26015da30 2024-09-10T20:58:23.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.254+0000 7fc267c01640 1 -- 172.21.15.98:0/3467588636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26015da30 msgr2=0x7fc2600cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.254+0000 7fc267c01640 1 --2- 172.21.15.98:0/3467588636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26015da30 0x7fc2600cea40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc25800c7a0 tx=0x7fc25800cc60 comp rx=0 tx=0).stop 2024-09-10T20:58:23.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.254+0000 7fc267c01640 1 -- 172.21.15.98:0/3467588636 shutdown_connections 2024-09-10T20:58:23.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.254+0000 7fc267c01640 1 --2- 172.21.15.98:0/3467588636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc26015da30 0x7fc2600cea40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.254+0000 7fc267c01640 1 --2- 172.21.15.98:0/3467588636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc26015b1e0 0x7fc2600d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.254+0000 7fc267c01640 1 --2- 172.21.15.98:0/3467588636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc26013cf80 0x7fc260150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.254+0000 7fc267c01640 1 -- 172.21.15.98:0/3467588636 >> 172.21.15.98:0/3467588636 conn(0x7fc2600b1760 msgr2=0x7fc2601598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.254+0000 7fc267c01640 1 -- 172.21.15.98:0/3467588636 shutdown_connections 2024-09-10T20:58:23.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.254+0000 7fc267c01640 1 -- 172.21.15.98:0/3467588636 wait complete. 2024-09-10T20:58:23.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.263+0000 7f6137c5a640 1 -- 172.21.15.98:0/3592259835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f6118005580 con 0x7f613016c760 2024-09-10T20:58:23.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.268+0000 7f61267fc640 1 -- 172.21.15.98:0/3592259835 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v16)=0 v16) ==== 119+0+0 (secure 0 0 0) 0x7f6120030d60 con 0x7f613016c760 2024-09-10T20:58:23.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.269+0000 7f6137c5a640 1 -- 172.21.15.98:0/3592259835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f613016c760 msgr2=0x7f61302016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.269+0000 7f6137c5a640 1 --2- 172.21.15.98:0/3592259835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f613016c760 0x7f61302016b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f612002a8f0 tx=0x7f61200305e0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.269+0000 7f6137c5a640 1 -- 172.21.15.98:0/3592259835 shutdown_connections 2024-09-10T20:58:23.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.269+0000 7f6137c5a640 1 --2- 172.21.15.98:0/3592259835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f613016c760 0x7f61302016b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.269+0000 7f6137c5a640 1 --2- 172.21.15.98:0/3592259835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61301f7650 0x7f613016c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.269+0000 7f6137c5a640 1 --2- 172.21.15.98:0/3592259835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61301f7270 0x7f61301f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.269+0000 7f6137c5a640 1 -- 172.21.15.98:0/3592259835 >> 172.21.15.98:0/3592259835 conn(0x7f61300af660 msgr2=0x7f613006e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.269+0000 7f6137c5a640 1 -- 172.21.15.98:0/3592259835 shutdown_connections 2024-09-10T20:58:23.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.269+0000 7f6137c5a640 1 -- 172.21.15.98:0/3592259835 wait complete. 2024-09-10T20:58:23.276 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi098') 2024-09-10T20:58:23.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.280+0000 7f2a8a89b640 1 -- 172.21.15.98:0/248638651 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f2a70005580 con 0x7f2a8406ccc0 2024-09-10T20:58:23.289 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi098') 2024-09-10T20:58:23.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.283+0000 7f641a7fc640 1 -- 172.21.15.98:0/2796974624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f6414005580 con 0x7f6434072560 2024-09-10T20:58:23.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.285+0000 7f2a80ff9640 1 -- 172.21.15.98:0/248638651 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f2a74030d40 con 0x7f2a8406ccc0 2024-09-10T20:58:23.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.286+0000 7f2a8a89b640 1 -- 172.21.15.98:0/248638651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a8406ccc0 msgr2=0x7f2a842019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.286+0000 7f2a8a89b640 1 --2- 172.21.15.98:0/248638651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a8406ccc0 0x7f2a842019a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009920 tx=0x7f2a74030570 comp rx=0 tx=0).stop 2024-09-10T20:58:23.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.286+0000 7f2a8a89b640 1 -- 172.21.15.98:0/248638651 shutdown_connections 2024-09-10T20:58:23.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.286+0000 7f2a8a89b640 1 --2- 172.21.15.98:0/248638651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a840b5cf0 0x7f2a84202b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.286+0000 7f2a8a89b640 1 --2- 172.21.15.98:0/248638651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a8406d0a0 0x7f2a842023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.286+0000 7f2a8a89b640 1 --2- 172.21.15.98:0/248638651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a8406ccc0 0x7f2a842019a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.286+0000 7f2a8a89b640 1 -- 172.21.15.98:0/248638651 >> 172.21.15.98:0/248638651 conn(0x7f2a8406e480 msgr2=0x7f2a840cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.286+0000 7f2a8a89b640 1 -- 172.21.15.98:0/248638651 shutdown_connections 2024-09-10T20:58:23.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.286+0000 7f2a8a89b640 1 -- 172.21.15.98:0/248638651 wait complete. 2024-09-10T20:58:23.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.287+0000 7f6430ff9640 1 -- 172.21.15.98:0/2796974624 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v18)=0 v18) ==== 125+0+0 (secure 0 0 0) 0x7f6424003b90 con 0x7f6434072560 2024-09-10T20:58:23.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.289+0000 7f641a7fc640 1 -- 172.21.15.98:0/2796974624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6434072560 msgr2=0x7f6434156260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.289+0000 7f641a7fc640 1 --2- 172.21.15.98:0/2796974624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6434072560 0x7f6434156260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f642400c4c0 tx=0x7f642400c980 comp rx=0 tx=0).stop 2024-09-10T20:58:23.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.289+0000 7f641a7fc640 1 -- 172.21.15.98:0/2796974624 shutdown_connections 2024-09-10T20:58:23.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.289+0000 7f641a7fc640 1 --2- 172.21.15.98:0/2796974624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64340b30b0 0x7f643408b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.289+0000 7f641a7fc640 1 --2- 172.21.15.98:0/2796974624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64340b2cd0 0x7f643408b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.289+0000 7f641a7fc640 1 --2- 172.21.15.98:0/2796974624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6434072560 0x7f6434156260 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.289+0000 7f641a7fc640 1 -- 172.21.15.98:0/2796974624 >> 172.21.15.98:0/2796974624 conn(0x7f64340cfaf0 msgr2=0x7f6434139110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.289+0000 7f641a7fc640 1 -- 172.21.15.98:0/2796974624 shutdown_connections 2024-09-10T20:58:23.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.289+0000 7f641a7fc640 1 -- 172.21.15.98:0/2796974624 wait complete. 2024-09-10T20:58:23.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.295+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/1702992180 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7ff188005580 con 0x7ff1a40b2fa0 2024-09-10T20:58:23.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.300+0000 7ff192ffd640 1 -- 172.21.15.98:0/1702992180 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7ff19400ce20 con 0x7ff1a40b2fa0 2024-09-10T20:58:23.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.301+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/1702992180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a40b2fa0 msgr2=0x7ff1a40aca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.301+0000 7ff1a8a6e640 1 --2- 172.21.15.98:0/1702992180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a40b2fa0 0x7ff1a40aca20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff19400b750 tx=0x7ff19400bc10 comp rx=0 tx=0).stop 2024-09-10T20:58:23.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.301+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/1702992180 shutdown_connections 2024-09-10T20:58:23.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.301+0000 7ff1a8a6e640 1 --2- 172.21.15.98:0/1702992180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a41477b0 0x7ff1a40c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.301+0000 7ff1a8a6e640 1 --2- 172.21.15.98:0/1702992180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1a40b2fa0 0x7ff1a40aca20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.301+0000 7ff1a8a6e640 1 --2- 172.21.15.98:0/1702992180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40ad890 0x7ff1a40ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.301+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/1702992180 >> 172.21.15.98:0/1702992180 conn(0x7ff1a40cfaf0 msgr2=0x7ff1a40cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.301+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/1702992180 shutdown_connections 2024-09-10T20:58:23.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.301+0000 7ff1a8a6e640 1 -- 172.21.15.98:0/1702992180 wait complete. 2024-09-10T20:58:23.317 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi098') 2024-09-10T20:58:23.318 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi098') 2024-09-10T20:58:23.318 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi098') 2024-09-10T20:58:23.318 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi098') 2024-09-10T20:58:23.318 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi098') 2024-09-10T20:58:23.319 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T20:58:23.319 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T20:58:23.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.455+0000 7fe441d94640 1 -- 172.21.15.98:0/1106871577 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe43c0b5ee0 msgr2=0x7fe43c0cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.455+0000 7fe441d94640 1 --2- 172.21.15.98:0/1106871577 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe43c0b5ee0 0x7fe43c0cccc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0b2e90 tx=0x7fe43402a980 comp rx=0 tx=0).stop 2024-09-10T20:58:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.455+0000 7fe441d94640 1 -- 172.21.15.98:0/1106871577 shutdown_connections 2024-09-10T20:58:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.455+0000 7fe441d94640 1 --2- 172.21.15.98:0/1106871577 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe43c0b5ee0 0x7fe43c0cccc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.455+0000 7fe441d94640 1 --2- 172.21.15.98:0/1106871577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe43c069480 0x7fe43c0b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.455+0000 7fe441d94640 1 --2- 172.21.15.98:0/1106871577 >> v2:172.21.15.190:3300/0 conn(0x7fe43c15f1d0 0x7fe43c06ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.455+0000 7fe441d94640 1 -- 172.21.15.98:0/1106871577 >> 172.21.15.98:0/1106871577 conn(0x7fe43c074170 msgr2=0x7fe43c074580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.455+0000 7fe441d94640 1 -- 172.21.15.98:0/1106871577 shutdown_connections 2024-09-10T20:58:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.455+0000 7fe441d94640 1 -- 172.21.15.98:0/1106871577 wait complete. 2024-09-10T20:58:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.456+0000 7fe441d94640 1 Processor -- start 2024-09-10T20:58:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.456+0000 7fe441d94640 1 -- start start 2024-09-10T20:58:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe441d94640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe43c069480 0x7fe43c0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe441d94640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe43c0b5ee0 0x7fe43c15b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe441d94640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe43c15f1d0 0x7fe43c06ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe441d94640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c0cd380 con 0x7fe43c0b5ee0 2024-09-10T20:58:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe441d94640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c0cd200 con 0x7fe43c069480 2024-09-10T20:58:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe441d94640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe43c0cd500 con 0x7fe43c15f1d0 2024-09-10T20:58:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe432ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe43c0b5ee0 0x7fe43c15b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe432ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe43c0b5ee0 0x7fe43c15b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49838/0 (socket says 172.21.15.98:49838) 2024-09-10T20:58:23.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe432ffd640 1 -- 172.21.15.98:0/2697134206 learned_addr learned my addr 172.21.15.98:0/2697134206 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:23.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe43b7fe640 1 --2- 172.21.15.98:0/2697134206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe43c069480 0x7fe43c0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe43bfff640 1 --2- 172.21.15.98:0/2697134206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe43c15f1d0 0x7fe43c06ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:23.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe432ffd640 1 -- 172.21.15.98:0/2697134206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe43c15f1d0 msgr2=0x7fe43c06ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe432ffd640 1 --2- 172.21.15.98:0/2697134206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe43c15f1d0 0x7fe43c06ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe432ffd640 1 -- 172.21.15.98:0/2697134206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe43c069480 msgr2=0x7fe43c0ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe432ffd640 1 --2- 172.21.15.98:0/2697134206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe43c069480 0x7fe43c0ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.457+0000 7fe432ffd640 1 -- 172.21.15.98:0/2697134206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c15baf0 con 0x7fe43c0b5ee0 2024-09-10T20:58:23.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.458+0000 7fe432ffd640 1 --2- 172.21.15.98:0/2697134206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe43c0b5ee0 0x7fe43c15b5b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe42400e9f0 tx=0x7fe42400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:23.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.458+0000 7fe438ff9640 1 -- 172.21.15.98:0/2697134206 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42400ccc0 con 0x7fe43c0b5ee0 2024-09-10T20:58:23.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.458+0000 7fe438ff9640 1 -- 172.21.15.98:0/2697134206 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe42400ce60 con 0x7fe43c0b5ee0 2024-09-10T20:58:23.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.458+0000 7fe441d94640 1 -- 172.21.15.98:0/2697134206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43c0adc40 con 0x7fe43c0b5ee0 2024-09-10T20:58:23.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.458+0000 7fe438ff9640 1 -- 172.21.15.98:0/2697134206 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424005260 con 0x7fe43c0b5ee0 2024-09-10T20:58:23.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.458+0000 7fe441d94640 1 -- 172.21.15.98:0/2697134206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43c0af960 con 0x7fe43c0b5ee0 2024-09-10T20:58:23.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.458+0000 7fe438ff9640 1 -- 172.21.15.98:0/2697134206 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fe4240047a0 con 0x7fe43c0b5ee0 2024-09-10T20:58:23.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.458+0000 7fe438ff9640 1 -- 172.21.15.98:0/2697134206 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe424018770 con 0x7fe43c0b5ee0 2024-09-10T20:58:23.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.459+0000 7fe441d94640 1 -- 172.21.15.98:0/2697134206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe43c0cda20 con 0x7fe43c0b5ee0 2024-09-10T20:58:23.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.461+0000 7fe438ff9640 1 -- 172.21.15.98:0/2697134206 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fe424038950 con 0x7fe43c0b5ee0 2024-09-10T20:58:23.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.525+0000 7fe441d94640 1 -- 172.21.15.98:0/2697134206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fe43c075dc0 con 0x7fe43c0b5ee0 2024-09-10T20:58:23.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.527+0000 7fe438ff9640 1 -- 172.21.15.98:0/2697134206 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fe424026030 con 0x7fe43c0b5ee0 2024-09-10T20:58:23.532 INFO:teuthology.orchestra.run.smithi098.stderr:adjusted tunables profile to default 2024-09-10T20:58:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.527+0000 7fe441d94640 1 -- 172.21.15.98:0/2697134206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe43c0b5ee0 msgr2=0x7fe43c15b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.527+0000 7fe441d94640 1 --2- 172.21.15.98:0/2697134206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe43c0b5ee0 0x7fe43c15b5b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe42400e9f0 tx=0x7fe42400eeb0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.527+0000 7fe441d94640 1 -- 172.21.15.98:0/2697134206 shutdown_connections 2024-09-10T20:58:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.527+0000 7fe441d94640 1 --2- 172.21.15.98:0/2697134206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe43c15f1d0 0x7fe43c06ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.527+0000 7fe441d94640 1 --2- 172.21.15.98:0/2697134206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe43c0b5ee0 0x7fe43c15b5b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.527+0000 7fe441d94640 1 --2- 172.21.15.98:0/2697134206 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe43c069480 0x7fe43c0ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.527+0000 7fe441d94640 1 -- 172.21.15.98:0/2697134206 >> 172.21.15.98:0/2697134206 conn(0x7fe43c074170 msgr2=0x7fe43c0706b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.528+0000 7fe441d94640 1 -- 172.21.15.98:0/2697134206 shutdown_connections 2024-09-10T20:58:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:23.528+0000 7fe441d94640 1 -- 172.21.15.98:0/2697134206 wait complete. 2024-09-10T20:58:23.543 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T20:58:23.543 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T20:58:23.544 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:23.544 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T20:58:23.568 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:23.568 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T20:58:23.634 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:23.634 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T20:58:23.699 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:23.700 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-10T20:58:23.764 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:58:23.764 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-10T20:58:23.789 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:58:23.790 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-10T20:58:23.854 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:58:23.854 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-10T20:58:23.919 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:58:23.919 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-10T20:58:23.984 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 3c2fd6ed-6940-4fc6-926a-85685cfdadd0 0 2024-09-10T20:58:24.125 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.121+0000 7f234d584640 1 -- 172.21.15.190:0/852821417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2348160530 msgr2=0x7f23480cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.121+0000 7f234d584640 1 --2- 172.21.15.190:0/852821417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2348160530 0x7f23480cf860 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2338005fd0 tx=0x7f233802a540 comp rx=0 tx=0).stop 2024-09-10T20:58:24.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.121+0000 7f234d584640 1 -- 172.21.15.190:0/852821417 shutdown_connections 2024-09-10T20:58:24.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.121+0000 7f234d584640 1 --2- 172.21.15.190:0/852821417 >> v2:172.21.15.98:3300/0 conn(0x7f23480b05c0 0x7f23480cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.121+0000 7f234d584640 1 --2- 172.21.15.190:0/852821417 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2348160530 0x7f23480cf860 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.121+0000 7f234d584640 1 -- 172.21.15.190:0/852821417 >> 172.21.15.190:0/852821417 conn(0x7f23480b37c0 msgr2=0x7f23480b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:24.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.121+0000 7f234d584640 1 -- 172.21.15.190:0/852821417 shutdown_connections 2024-09-10T20:58:24.126 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.121+0000 7f234d584640 1 -- 172.21.15.190:0/852821417 wait complete. 2024-09-10T20:58:24.130 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.123+0000 7f234d584640 1 Processor -- start 2024-09-10T20:58:24.131 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.123+0000 7f234d584640 1 -- start start 2024-09-10T20:58:24.131 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.123+0000 7f234d584640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23480b05c0 0x7f234815e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.131 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.123+0000 7f234d584640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2348160530 0x7f234814a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.131 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.123+0000 7f234d584640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234814a850 0x7f234814c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.131 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.123+0000 7f234d584640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f23480b1990 con 0x7f23480b05c0 2024-09-10T20:58:24.131 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.123+0000 7f234d584640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f23480b34f0 con 0x7f2348160530 2024-09-10T20:58:24.132 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.123+0000 7f234d584640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f23480b1b10 con 0x7f234814a850 2024-09-10T20:58:24.132 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f23477fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234814a850 0x7f234814c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.132 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f23467fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2348160530 0x7f234814a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.132 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f23477fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234814a850 0x7f234814c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:43846/0 (socket says 172.21.15.190:43846) 2024-09-10T20:58:24.132 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f23477fe640 1 -- 172.21.15.190:0/1598244525 learned_addr learned my addr 172.21.15.190:0/1598244525 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-10T20:58:24.132 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f2346ffd640 1 --2- 172.21.15.190:0/1598244525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23480b05c0 0x7f234815e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.133 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f23467fc640 1 -- 172.21.15.190:0/1598244525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234814a850 msgr2=0x7f234814c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.133 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f23467fc640 1 --2- 172.21.15.190:0/1598244525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234814a850 0x7f234814c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.133 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f23467fc640 1 -- 172.21.15.190:0/1598244525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23480b05c0 msgr2=0x7f234815e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.133 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f23467fc640 1 --2- 172.21.15.190:0/1598244525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23480b05c0 0x7f234815e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.134 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f23467fc640 1 -- 172.21.15.190:0/1598244525 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234814ce20 con 0x7f2348160530 2024-09-10T20:58:24.134 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f23477fe640 1 --2- 172.21.15.190:0/1598244525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234814a850 0x7f234814c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:58:24.134 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f23467fc640 1 --2- 172.21.15.190:0/1598244525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2348160530 0x7f234814a310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f23480747e0 tx=0x7f233000c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:24.134 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f233f7fe640 1 -- 172.21.15.190:0/1598244525 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330017070 con 0x7f2348160530 2024-09-10T20:58:24.135 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f233f7fe640 1 -- 172.21.15.190:0/1598244525 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23300053c0 con 0x7f2348160530 2024-09-10T20:58:24.135 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f233f7fe640 1 -- 172.21.15.190:0/1598244525 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330003850 con 0x7f2348160530 2024-09-10T20:58:24.135 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f234d584640 1 -- 172.21.15.190:0/1598244525 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234814d100 con 0x7f2348160530 2024-09-10T20:58:24.135 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.124+0000 7f234d584640 1 -- 172.21.15.190:0/1598244525 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23480caef0 con 0x7f2348160530 2024-09-10T20:58:24.135 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.125+0000 7f233f7fe640 1 -- 172.21.15.190:0/1598244525 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f23300039f0 con 0x7f2348160530 2024-09-10T20:58:24.136 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.125+0000 7f233f7fe640 1 -- 172.21.15.190:0/1598244525 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f233001c780 con 0x7f2348160530 2024-09-10T20:58:24.136 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.125+0000 7f234d584640 1 -- 172.21.15.190:0/1598244525 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23480cb0f0 con 0x7f2348160530 2024-09-10T20:58:24.136 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.127+0000 7f233f7fe640 1 -- 172.21.15.190:0/1598244525 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f2330021080 con 0x7f2348160530 2024-09-10T20:58:24.202 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.198+0000 7f234d584640 1 -- 172.21.15.190:0/1598244525 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3c2fd6ed-6940-4fc6-926a-85685cfdadd0", "id": 0} v 0) -- 0x7f23480b2000 con 0x7f2348160530 2024-09-10T20:58:24.205 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.201+0000 7f233f7fe640 1 -- 172.21.15.190:0/1598244525 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3c2fd6ed-6940-4fc6-926a-85685cfdadd0", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f233001c9d0 con 0x7f2348160530 2024-09-10T20:58:24.205 INFO:teuthology.orchestra.run.smithi190.stdout:0 2024-09-10T20:58:24.206 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.202+0000 7f234d584640 1 -- 172.21.15.190:0/1598244525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2348160530 msgr2=0x7f234814a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.206 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.202+0000 7f234d584640 1 --2- 172.21.15.190:0/1598244525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2348160530 0x7f234814a310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f23480747e0 tx=0x7f233000c820 comp rx=0 tx=0).stop 2024-09-10T20:58:24.206 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.202+0000 7f234d584640 1 -- 172.21.15.190:0/1598244525 shutdown_connections 2024-09-10T20:58:24.206 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.202+0000 7f234d584640 1 --2- 172.21.15.190:0/1598244525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f234814a850 0x7f234814c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.202+0000 7f234d584640 1 --2- 172.21.15.190:0/1598244525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2348160530 0x7f234814a310 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.202+0000 7f234d584640 1 --2- 172.21.15.190:0/1598244525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23480b05c0 0x7f234815e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.202+0000 7f234d584640 1 -- 172.21.15.190:0/1598244525 >> 172.21.15.190:0/1598244525 conn(0x7f23480b37c0 msgr2=0x7f23480cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:24.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.202+0000 7f234d584640 1 -- 172.21.15.190:0/1598244525 shutdown_connections 2024-09-10T20:58:24.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.202+0000 7f234d584640 1 -- 172.21.15.190:0/1598244525 wait complete. 2024-09-10T20:58:24.216 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 05006fe3-bbb6-4206-916e-43ff3ea7e711 1 2024-09-10T20:58:24.318 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.314+0000 7f4d1c99b640 1 -- 172.21.15.190:0/1141701673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1815be00 msgr2=0x7f4d18070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.314+0000 7f4d1c99b640 1 --2- 172.21.15.190:0/1141701673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1815be00 0x7f4d18070760 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4d180ccb30 tx=0x7f4d0802a670 comp rx=0 tx=0).stop 2024-09-10T20:58:24.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.314+0000 7f4d1c99b640 1 -- 172.21.15.190:0/1141701673 shutdown_connections 2024-09-10T20:58:24.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.314+0000 7f4d1c99b640 1 --2- 172.21.15.190:0/1141701673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1815be00 0x7f4d18070760 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.314+0000 7f4d1c99b640 1 -- 172.21.15.190:0/1141701673 >> 172.21.15.190:0/1141701673 conn(0x7f4d180d1d10 msgr2=0x7f4d180d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:24.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.314+0000 7f4d1c99b640 1 -- 172.21.15.190:0/1141701673 shutdown_connections 2024-09-10T20:58:24.319 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.314+0000 7f4d1c99b640 1 -- 172.21.15.190:0/1141701673 wait complete. 2024-09-10T20:58:24.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.316+0000 7f4d1c99b640 1 Processor -- start 2024-09-10T20:58:24.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.316+0000 7f4d1c99b640 1 -- start start 2024-09-10T20:58:24.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.316+0000 7f4d1c99b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d180d0a00 0x7f4d181521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.316+0000 7f4d1c99b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d180d2830 0x7f4d18151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.316+0000 7f4d1c99b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1815be00 0x7f4d180cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.316+0000 7f4d1c99b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4d180cfc70 con 0x7f4d180d2830 2024-09-10T20:58:24.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.316+0000 7f4d1c99b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4d180cfaf0 con 0x7f4d1815be00 2024-09-10T20:58:24.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.316+0000 7f4d1c99b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4d180cfdf0 con 0x7f4d180d0a00 2024-09-10T20:58:24.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.316+0000 7f4d1659c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d180d0a00 0x7f4d181521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.316+0000 7f4d16d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1815be00 0x7f4d180cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.316+0000 7f4d16d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1815be00 0x7f4d180cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:54770/0 (socket says 172.21.15.190:54770) 2024-09-10T20:58:24.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.316+0000 7f4d1659c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d180d0a00 0x7f4d181521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:43850/0 (socket says 172.21.15.190:43850) 2024-09-10T20:58:24.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.316+0000 7f4d16d9d640 1 -- 172.21.15.190:0/1308302033 learned_addr learned my addr 172.21.15.190:0/1308302033 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-10T20:58:24.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.316+0000 7f4d0ffff640 1 --2- 172.21.15.190:0/1308302033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d180d2830 0x7f4d18151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.317+0000 7f4d1659c640 1 -- 172.21.15.190:0/1308302033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1815be00 msgr2=0x7f4d180cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.317+0000 7f4d1659c640 1 --2- 172.21.15.190:0/1308302033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1815be00 0x7f4d180cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.317+0000 7f4d1659c640 1 -- 172.21.15.190:0/1308302033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d180d2830 msgr2=0x7f4d18151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.317+0000 7f4d1659c640 1 --2- 172.21.15.190:0/1308302033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d180d2830 0x7f4d18151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.317+0000 7f4d1659c640 1 -- 172.21.15.190:0/1308302033 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d180c5bb0 con 0x7f4d180d0a00 2024-09-10T20:58:24.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.317+0000 7f4d16d9d640 1 --2- 172.21.15.190:0/1308302033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1815be00 0x7f4d180cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:58:24.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.317+0000 7f4d1659c640 1 --2- 172.21.15.190:0/1308302033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d180d0a00 0x7f4d181521f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4d08031000 tx=0x7f4d080039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:24.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.317+0000 7f4d0effd640 1 -- 172.21.15.190:0/1308302033 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0803e030 con 0x7f4d180d0a00 2024-09-10T20:58:24.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.317+0000 7f4d0effd640 1 -- 172.21.15.190:0/1308302033 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d080047a0 con 0x7f4d180d0a00 2024-09-10T20:58:24.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.317+0000 7f4d1c99b640 1 -- 172.21.15.190:0/1308302033 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d180c5e90 con 0x7f4d180d0a00 2024-09-10T20:58:24.323 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.317+0000 7f4d0effd640 1 -- 172.21.15.190:0/1308302033 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08035040 con 0x7f4d180d0a00 2024-09-10T20:58:24.324 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.317+0000 7f4d1c99b640 1 -- 172.21.15.190:0/1308302033 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d180c6450 con 0x7f4d180d0a00 2024-09-10T20:58:24.324 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.317+0000 7f4d0effd640 1 -- 172.21.15.190:0/1308302033 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f4d08004f40 con 0x7f4d180d0a00 2024-09-10T20:58:24.324 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.317+0000 7f4d0effd640 1 -- 172.21.15.190:0/1308302033 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f4d08031400 con 0x7f4d180d0a00 2024-09-10T20:58:24.324 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.318+0000 7f4d1c99b640 1 -- 172.21.15.190:0/1308302033 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d180cd880 con 0x7f4d180d0a00 2024-09-10T20:58:24.326 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.322+0000 7f4d0effd640 1 -- 172.21.15.190:0/1308302033 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f4d08043410 con 0x7f4d180d0a00 2024-09-10T20:58:24.396 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.393+0000 7f4d1c99b640 1 -- 172.21.15.190:0/1308302033 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "05006fe3-bbb6-4206-916e-43ff3ea7e711", "id": 1} v 0) -- 0x7f4d180b7c90 con 0x7f4d180d0a00 2024-09-10T20:58:24.400 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.397+0000 7f4d0effd640 1 -- 172.21.15.190:0/1308302033 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "05006fe3-bbb6-4206-916e-43ff3ea7e711", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f4d080435f0 con 0x7f4d180d0a00 2024-09-10T20:58:24.401 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-09-10T20:58:24.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.398+0000 7f4d1c99b640 1 -- 172.21.15.190:0/1308302033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d180d0a00 msgr2=0x7f4d181521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.398+0000 7f4d1c99b640 1 --2- 172.21.15.190:0/1308302033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d180d0a00 0x7f4d181521f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4d08031000 tx=0x7f4d080039e0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.398+0000 7f4d1c99b640 1 -- 172.21.15.190:0/1308302033 shutdown_connections 2024-09-10T20:58:24.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.398+0000 7f4d1c99b640 1 --2- 172.21.15.190:0/1308302033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1815be00 0x7f4d180cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.398+0000 7f4d1c99b640 1 --2- 172.21.15.190:0/1308302033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d180d2830 0x7f4d18151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.398+0000 7f4d1c99b640 1 --2- 172.21.15.190:0/1308302033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d180d0a00 0x7f4d181521f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.398+0000 7f4d1c99b640 1 -- 172.21.15.190:0/1308302033 >> 172.21.15.190:0/1308302033 conn(0x7f4d180d1d10 msgr2=0x7f4d180b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:24.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.398+0000 7f4d1c99b640 1 -- 172.21.15.190:0/1308302033 shutdown_connections 2024-09-10T20:58:24.402 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.398+0000 7f4d1c99b640 1 -- 172.21.15.190:0/1308302033 wait complete. 2024-09-10T20:58:24.412 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new ba3940dc-1a8e-4ca7-a66d-135616696e32 2 2024-09-10T20:58:24.513 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.509+0000 7fe70067f640 1 -- 172.21.15.190:0/2228042595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6f806cf90 msgr2=0x7fe6f81416d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.514 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.509+0000 7fe70067f640 1 --2- 172.21.15.190:0/2228042595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6f806cf90 0x7fe6f81416d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8070130 tx=0x7fe6f002aac0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.514 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.509+0000 7fe70067f640 1 -- 172.21.15.190:0/2228042595 shutdown_connections 2024-09-10T20:58:24.514 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.509+0000 7fe70067f640 1 --2- 172.21.15.190:0/2228042595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6f806cf90 0x7fe6f81416d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.514 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.509+0000 7fe70067f640 1 --2- 172.21.15.190:0/2228042595 >> v2:172.21.15.98:3300/0 conn(0x7fe6f80d02d0 0x7fe6f806ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.514 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.509+0000 7fe70067f640 1 -- 172.21.15.190:0/2228042595 >> 172.21.15.190:0/2228042595 conn(0x7fe6f806e480 msgr2=0x7fe6f80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:24.514 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.509+0000 7fe70067f640 1 -- 172.21.15.190:0/2228042595 shutdown_connections 2024-09-10T20:58:24.515 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.509+0000 7fe70067f640 1 -- 172.21.15.190:0/2228042595 wait complete. 2024-09-10T20:58:24.517 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.510+0000 7fe70067f640 1 Processor -- start 2024-09-10T20:58:24.517 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.511+0000 7fe70067f640 1 -- start start 2024-09-10T20:58:24.517 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.511+0000 7fe70067f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f806cf90 0x7fe6f8156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.511+0000 7fe70067f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6f8074520 0x7fe6f80a9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.511+0000 7fe70067f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6f80d02d0 0x7fe6f80a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.511+0000 7fe70067f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f815c390 con 0x7fe6f806cf90 2024-09-10T20:58:24.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.511+0000 7fe70067f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f815c210 con 0x7fe6f80d02d0 2024-09-10T20:58:24.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.511+0000 7fe70067f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f8141c10 con 0x7fe6f8074520 2024-09-10T20:58:24.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.511+0000 7fe6fec1c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6f80d02d0 0x7fe6f80a9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.511+0000 7fe6fdc1a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6f8074520 0x7fe6f80a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.511+0000 7fe6fe41b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f806cf90 0x7fe6f8156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.511+0000 7fe6fdc1a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6f8074520 0x7fe6f80a9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:43856/0 (socket says 172.21.15.190:43856) 2024-09-10T20:58:24.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.511+0000 7fe6fec1c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6f80d02d0 0x7fe6f80a9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:54782/0 (socket says 172.21.15.190:54782) 2024-09-10T20:58:24.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.511+0000 7fe6fdc1a640 1 -- 172.21.15.190:0/2798557118 learned_addr learned my addr 172.21.15.190:0/2798557118 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-10T20:58:24.518 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe6fe41b640 1 -- 172.21.15.190:0/2798557118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6f8074520 msgr2=0x7fe6f80a9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe6fe41b640 1 --2- 172.21.15.190:0/2798557118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6f8074520 0x7fe6f80a9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe6fe41b640 1 -- 172.21.15.190:0/2798557118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6f80d02d0 msgr2=0x7fe6f80a9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe6fe41b640 1 --2- 172.21.15.190:0/2798557118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6f80d02d0 0x7fe6f80a9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe6fe41b640 1 -- 172.21.15.190:0/2798557118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f81572e0 con 0x7fe6f806cf90 2024-09-10T20:58:24.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe6fec1c640 1 --2- 172.21.15.190:0/2798557118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6f80d02d0 0x7fe6f80a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:58:24.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe6fdc1a640 1 --2- 172.21.15.190:0/2798557118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6f8074520 0x7fe6f80a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:58:24.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe6fe41b640 1 --2- 172.21.15.190:0/2798557118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f806cf90 0x7fe6f8156840 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400c3c0 tx=0x7fe6f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:24.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe6eaffd640 1 -- 172.21.15.190:0/2798557118 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4017070 con 0x7fe6f806cf90 2024-09-10T20:58:24.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe6eaffd640 1 -- 172.21.15.190:0/2798557118 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6f40053c0 con 0x7fe6f806cf90 2024-09-10T20:58:24.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe70067f640 1 -- 172.21.15.190:0/2798557118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f81575c0 con 0x7fe6f806cf90 2024-09-10T20:58:24.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe6eaffd640 1 -- 172.21.15.190:0/2798557118 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4003850 con 0x7fe6f806cf90 2024-09-10T20:58:24.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe70067f640 1 -- 172.21.15.190:0/2798557118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f8157b80 con 0x7fe6f806cf90 2024-09-10T20:58:24.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe6eaffd640 1 -- 172.21.15.190:0/2798557118 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fe6f40039f0 con 0x7fe6f806cf90 2024-09-10T20:58:24.519 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.512+0000 7fe6eaffd640 1 -- 172.21.15.190:0/2798557118 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fe6f4012840 con 0x7fe6f806cf90 2024-09-10T20:58:24.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.513+0000 7fe70067f640 1 -- 172.21.15.190:0/2798557118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f8157fe0 con 0x7fe6f806cf90 2024-09-10T20:58:24.520 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.515+0000 7fe6eaffd640 1 -- 172.21.15.190:0/2798557118 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fe6f4021070 con 0x7fe6f806cf90 2024-09-10T20:58:24.592 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.588+0000 7fe70067f640 1 -- 172.21.15.190:0/2798557118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ba3940dc-1a8e-4ca7-a66d-135616696e32", "id": 2} v 0) -- 0x7fe6f81420e0 con 0x7fe6f806cf90 2024-09-10T20:58:24.594 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.590+0000 7fe6eaffd640 1 -- 172.21.15.190:0/2798557118 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ba3940dc-1a8e-4ca7-a66d-135616696e32", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fe6f4012420 con 0x7fe6f806cf90 2024-09-10T20:58:24.594 INFO:teuthology.orchestra.run.smithi190.stdout:2 2024-09-10T20:58:24.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.591+0000 7fe70067f640 1 -- 172.21.15.190:0/2798557118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f806cf90 msgr2=0x7fe6f8156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.591+0000 7fe70067f640 1 --2- 172.21.15.190:0/2798557118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f806cf90 0x7fe6f8156840 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400c3c0 tx=0x7fe6f400c880 comp rx=0 tx=0).stop 2024-09-10T20:58:24.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.591+0000 7fe70067f640 1 -- 172.21.15.190:0/2798557118 shutdown_connections 2024-09-10T20:58:24.595 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.591+0000 7fe70067f640 1 --2- 172.21.15.190:0/2798557118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6f80d02d0 0x7fe6f80a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.596 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.591+0000 7fe70067f640 1 --2- 172.21.15.190:0/2798557118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6f8074520 0x7fe6f80a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.596 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.591+0000 7fe70067f640 1 --2- 172.21.15.190:0/2798557118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f806cf90 0x7fe6f8156840 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.596 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.591+0000 7fe70067f640 1 -- 172.21.15.190:0/2798557118 >> 172.21.15.190:0/2798557118 conn(0x7fe6f806e480 msgr2=0x7fe6f813e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:24.596 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.591+0000 7fe70067f640 1 -- 172.21.15.190:0/2798557118 shutdown_connections 2024-09-10T20:58:24.596 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.591+0000 7fe70067f640 1 -- 172.21.15.190:0/2798557118 wait complete. 2024-09-10T20:58:24.605 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 686f1fed-2a52-4bcd-b02b-1e682946f04b 3 2024-09-10T20:58:24.706 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.702+0000 7fc284633640 1 -- 172.21.15.190:0/3416577751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27c15f6a0 msgr2=0x7fc27c0ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.707 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.702+0000 7fc284633640 1 --2- 172.21.15.190:0/3416577751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27c15f6a0 0x7fc27c0ce5e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc274005fd0 tx=0x7fc27402a820 comp rx=0 tx=0).stop 2024-09-10T20:58:24.707 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.702+0000 7fc284633640 1 -- 172.21.15.190:0/3416577751 shutdown_connections 2024-09-10T20:58:24.707 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.702+0000 7fc284633640 1 --2- 172.21.15.190:0/3416577751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27c15f6a0 0x7fc27c0ce5e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.707 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.702+0000 7fc284633640 1 --2- 172.21.15.190:0/3416577751 >> v2:172.21.15.190:3300/0 conn(0x7fc27c0cfb30 0x7fc27c0ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.707 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.702+0000 7fc284633640 1 -- 172.21.15.190:0/3416577751 >> 172.21.15.190:0/3416577751 conn(0x7fc27c0d1bc0 msgr2=0x7fc27c0d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:24.707 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.702+0000 7fc284633640 1 -- 172.21.15.190:0/3416577751 shutdown_connections 2024-09-10T20:58:24.707 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.702+0000 7fc284633640 1 -- 172.21.15.190:0/3416577751 wait complete. 2024-09-10T20:58:24.710 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.703+0000 7fc284633640 1 Processor -- start 2024-09-10T20:58:24.710 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.703+0000 7fc284633640 1 -- start start 2024-09-10T20:58:24.710 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.704+0000 7fc284633640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27c15f2c0 0x7fc27c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.710 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.704+0000 7fc284633640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc27c15f6a0 0x7fc27c0b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.710 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.704+0000 7fc284633640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc27c15bd20 0x7fc27c15c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.710 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.704+0000 7fc284633640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c0cce10 con 0x7fc27c15f6a0 2024-09-10T20:58:24.711 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.704+0000 7fc284633640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c0ceb20 con 0x7fc27c15bd20 2024-09-10T20:58:24.711 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.704+0000 7fc284633640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc27c15c8c0 con 0x7fc27c15f2c0 2024-09-10T20:58:24.711 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.704+0000 7fc282bd0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc27c15bd20 0x7fc27c15c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.711 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.704+0000 7fc2823cf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27c15f2c0 0x7fc27c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.711 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.704+0000 7fc281bce640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc27c15f6a0 0x7fc27c0b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.711 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.704+0000 7fc282bd0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc27c15bd20 0x7fc27c15c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:54796/0 (socket says 172.21.15.190:54796) 2024-09-10T20:58:24.711 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.704+0000 7fc282bd0640 1 -- 172.21.15.190:0/2227109053 learned_addr learned my addr 172.21.15.190:0/2227109053 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-10T20:58:24.711 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.704+0000 7fc2823cf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27c15f2c0 0x7fc27c154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:43880/0 (socket says 172.21.15.190:43880) 2024-09-10T20:58:24.711 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.705+0000 7fc282bd0640 1 -- 172.21.15.190:0/2227109053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27c15f2c0 msgr2=0x7fc27c154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.711 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.705+0000 7fc282bd0640 1 --2- 172.21.15.190:0/2227109053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27c15f2c0 0x7fc27c154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.711 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.705+0000 7fc282bd0640 1 -- 172.21.15.190:0/2227109053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc27c15f6a0 msgr2=0x7fc27c0b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.711 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.705+0000 7fc282bd0640 1 --2- 172.21.15.190:0/2227109053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc27c15f6a0 0x7fc27c0b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.711 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.705+0000 7fc282bd0640 1 -- 172.21.15.190:0/2227109053 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27c0c6b10 con 0x7fc27c15bd20 2024-09-10T20:58:24.712 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.705+0000 7fc281bce640 1 --2- 172.21.15.190:0/2227109053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc27c15f6a0 0x7fc27c0b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:58:24.712 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.705+0000 7fc282bd0640 1 --2- 172.21.15.190:0/2227109053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc27c15bd20 0x7fc27c15c1a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc274006010 tx=0x7fc274032be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:24.712 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.705+0000 7fc272ffd640 1 -- 172.21.15.190:0/2227109053 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274037040 con 0x7fc27c15bd20 2024-09-10T20:58:24.712 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.705+0000 7fc272ffd640 1 -- 172.21.15.190:0/2227109053 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc274042070 con 0x7fc27c15bd20 2024-09-10T20:58:24.712 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.705+0000 7fc284633640 1 -- 172.21.15.190:0/2227109053 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27c0c6df0 con 0x7fc27c15bd20 2024-09-10T20:58:24.712 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.705+0000 7fc272ffd640 1 -- 172.21.15.190:0/2227109053 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274004040 con 0x7fc27c15bd20 2024-09-10T20:58:24.712 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.705+0000 7fc284633640 1 -- 172.21.15.190:0/2227109053 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27c0c73b0 con 0x7fc27c15bd20 2024-09-10T20:58:24.712 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.705+0000 7fc272ffd640 1 -- 172.21.15.190:0/2227109053 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fc27403a030 con 0x7fc27c15bd20 2024-09-10T20:58:24.712 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.705+0000 7fc272ffd640 1 -- 172.21.15.190:0/2227109053 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fc274047990 con 0x7fc27c15bd20 2024-09-10T20:58:24.712 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.706+0000 7fc284633640 1 -- 172.21.15.190:0/2227109053 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc27c0cf0b0 con 0x7fc27c15bd20 2024-09-10T20:58:24.713 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.708+0000 7fc272ffd640 1 -- 172.21.15.190:0/2227109053 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fc274032450 con 0x7fc27c15bd20 2024-09-10T20:58:24.781 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.777+0000 7fc284633640 1 -- 172.21.15.190:0/2227109053 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "686f1fed-2a52-4bcd-b02b-1e682946f04b", "id": 3} v 0) -- 0x7fc27c0ce470 con 0x7fc27c15bd20 2024-09-10T20:58:24.784 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.780+0000 7fc272ffd640 1 -- 172.21.15.190:0/2227109053 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "686f1fed-2a52-4bcd-b02b-1e682946f04b", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fc274047500 con 0x7fc27c15bd20 2024-09-10T20:58:24.784 INFO:teuthology.orchestra.run.smithi190.stdout:3 2024-09-10T20:58:24.785 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.781+0000 7fc284633640 1 -- 172.21.15.190:0/2227109053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc27c15bd20 msgr2=0x7fc27c15c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.785 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.781+0000 7fc284633640 1 --2- 172.21.15.190:0/2227109053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc27c15bd20 0x7fc27c15c1a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc274006010 tx=0x7fc274032be0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.785 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.781+0000 7fc284633640 1 -- 172.21.15.190:0/2227109053 shutdown_connections 2024-09-10T20:58:24.786 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.781+0000 7fc284633640 1 --2- 172.21.15.190:0/2227109053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc27c15bd20 0x7fc27c15c1a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.786 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.781+0000 7fc284633640 1 --2- 172.21.15.190:0/2227109053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc27c15f6a0 0x7fc27c0b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.786 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.781+0000 7fc284633640 1 --2- 172.21.15.190:0/2227109053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27c15f2c0 0x7fc27c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.786 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.781+0000 7fc284633640 1 -- 172.21.15.190:0/2227109053 >> 172.21.15.190:0/2227109053 conn(0x7fc27c0d1bc0 msgr2=0x7fc27c1558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:24.786 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.781+0000 7fc284633640 1 -- 172.21.15.190:0/2227109053 shutdown_connections 2024-09-10T20:58:24.786 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.781+0000 7fc284633640 1 -- 172.21.15.190:0/2227109053 wait complete. 2024-09-10T20:58:24.795 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new ca6cdff1-7f33-4941-a075-71ff47a38b05 4 2024-09-10T20:58:24.896 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.892+0000 7fefcb441640 1 -- 172.21.15.190:0/3751835558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefc415e470 msgr2=0x7fefc406ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.897 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.892+0000 7fefcb441640 1 --2- 172.21.15.190:0/3751835558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefc415e470 0x7fefc406ca50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fefc0005fd0 tx=0x7fefc002a4b0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.897 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.892+0000 7fefcb441640 1 -- 172.21.15.190:0/3751835558 shutdown_connections 2024-09-10T20:58:24.897 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.892+0000 7fefcb441640 1 --2- 172.21.15.190:0/3751835558 >> v2:172.21.15.98:3300/0 conn(0x7fefc4074c90 0x7fefc40ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.897 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.892+0000 7fefcb441640 1 --2- 172.21.15.190:0/3751835558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefc415e470 0x7fefc406ca50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.897 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.892+0000 7fefcb441640 1 -- 172.21.15.190:0/3751835558 >> 172.21.15.190:0/3751835558 conn(0x7fefc4074170 msgr2=0x7fefc4074580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:24.897 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.892+0000 7fefcb441640 1 -- 172.21.15.190:0/3751835558 shutdown_connections 2024-09-10T20:58:24.897 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.892+0000 7fefcb441640 1 -- 172.21.15.190:0/3751835558 wait complete. 2024-09-10T20:58:24.899 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.893+0000 7fefcb441640 1 Processor -- start 2024-09-10T20:58:24.899 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.893+0000 7fefcb441640 1 -- start start 2024-09-10T20:58:24.899 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefcb441640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefc4074c90 0x7fefc41f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.899 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefcb441640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefc40ccbd0 0x7fefc4070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefcb441640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefc415e470 0x7fefc4070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:24.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefcb441640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fefc40cd080 con 0x7fefc415e470 2024-09-10T20:58:24.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefcb441640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fefc40afce0 con 0x7fefc4074c90 2024-09-10T20:58:24.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefcb441640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fefc40cd200 con 0x7fefc40ccbd0 2024-09-10T20:58:24.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefc89dc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefc40ccbd0 0x7fefc4070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefc91dd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefc4074c90 0x7fefc41f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefc89dc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefc40ccbd0 0x7fefc4070030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:43892/0 (socket says 172.21.15.190:43892) 2024-09-10T20:58:24.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefc89dc640 1 -- 172.21.15.190:0/2869328104 learned_addr learned my addr 172.21.15.190:0/2869328104 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-10T20:58:24.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefc99de640 1 --2- 172.21.15.190:0/2869328104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefc415e470 0x7fefc4070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:24.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefc89dc640 1 -- 172.21.15.190:0/2869328104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefc4074c90 msgr2=0x7fefc41f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefc89dc640 1 --2- 172.21.15.190:0/2869328104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefc4074c90 0x7fefc41f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefc89dc640 1 -- 172.21.15.190:0/2869328104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefc415e470 msgr2=0x7fefc4070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefc89dc640 1 --2- 172.21.15.190:0/2869328104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefc415e470 0x7fefc4070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.894+0000 7fefc89dc640 1 -- 172.21.15.190:0/2869328104 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc41f6a20 con 0x7fefc40ccbd0 2024-09-10T20:58:24.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.895+0000 7fefc89dc640 1 --2- 172.21.15.190:0/2869328104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefc40ccbd0 0x7fefc4070030 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fefb800e9f0 tx=0x7fefb800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:24.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.895+0000 7fefb5ffb640 1 -- 172.21.15.190:0/2869328104 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb800cd80 con 0x7fefc40ccbd0 2024-09-10T20:58:24.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.895+0000 7fefb5ffb640 1 -- 172.21.15.190:0/2869328104 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefb8004020 con 0x7fefc40ccbd0 2024-09-10T20:58:24.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.895+0000 7fefcb441640 1 -- 172.21.15.190:0/2869328104 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc41f5c60 con 0x7fefc40ccbd0 2024-09-10T20:58:24.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.895+0000 7fefb5ffb640 1 -- 172.21.15.190:0/2869328104 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8004430 con 0x7fefc40ccbd0 2024-09-10T20:58:24.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.895+0000 7fefcb441640 1 -- 172.21.15.190:0/2869328104 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc41f6220 con 0x7fefc40ccbd0 2024-09-10T20:58:24.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.895+0000 7fefb5ffb640 1 -- 172.21.15.190:0/2869328104 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fefb8003980 con 0x7fefc40ccbd0 2024-09-10T20:58:24.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.895+0000 7fefb5ffb640 1 -- 172.21.15.190:0/2869328104 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fefb8018940 con 0x7fefc40ccbd0 2024-09-10T20:58:24.902 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.896+0000 7fefcb441640 1 -- 172.21.15.190:0/2869328104 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc40ca9a0 con 0x7fefc40ccbd0 2024-09-10T20:58:24.902 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.898+0000 7fefb5ffb640 1 -- 172.21.15.190:0/2869328104 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fefb8026070 con 0x7fefc40ccbd0 2024-09-10T20:58:24.971 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.967+0000 7fefcb441640 1 -- 172.21.15.190:0/2869328104 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ca6cdff1-7f33-4941-a075-71ff47a38b05", "id": 4} v 0) -- 0x7fefc40b01f0 con 0x7fefc40ccbd0 2024-09-10T20:58:24.974 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.971+0000 7fefb5ffb640 1 -- 172.21.15.190:0/2869328104 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ca6cdff1-7f33-4941-a075-71ff47a38b05", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fefb8003ca0 con 0x7fefc40ccbd0 2024-09-10T20:58:24.975 INFO:teuthology.orchestra.run.smithi190.stdout:4 2024-09-10T20:58:24.976 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.972+0000 7fefcb441640 1 -- 172.21.15.190:0/2869328104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefc40ccbd0 msgr2=0x7fefc4070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:24.976 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.972+0000 7fefcb441640 1 --2- 172.21.15.190:0/2869328104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefc40ccbd0 0x7fefc4070030 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fefb800e9f0 tx=0x7fefb800eeb0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.976 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.972+0000 7fefcb441640 1 -- 172.21.15.190:0/2869328104 shutdown_connections 2024-09-10T20:58:24.976 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.972+0000 7fefcb441640 1 --2- 172.21.15.190:0/2869328104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefc415e470 0x7fefc4070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.976 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.972+0000 7fefcb441640 1 --2- 172.21.15.190:0/2869328104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fefc40ccbd0 0x7fefc4070030 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.976 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.972+0000 7fefcb441640 1 --2- 172.21.15.190:0/2869328104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fefc4074c90 0x7fefc41f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:24.977 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.972+0000 7fefcb441640 1 -- 172.21.15.190:0/2869328104 >> 172.21.15.190:0/2869328104 conn(0x7fefc4074170 msgr2=0x7fefc40d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:24.977 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.972+0000 7fefcb441640 1 -- 172.21.15.190:0/2869328104 shutdown_connections 2024-09-10T20:58:24.977 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:24.972+0000 7fefcb441640 1 -- 172.21.15.190:0/2869328104 wait complete. 2024-09-10T20:58:24.986 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 914f0013-7767-489d-aeaf-44f67aeb6810 5 2024-09-10T20:58:25.087 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.083+0000 7f5fd895d640 1 -- 172.21.15.190:0/4120017559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fd415eef0 msgr2=0x7f5fd415b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.087 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.083+0000 7f5fd895d640 1 --2- 172.21.15.190:0/4120017559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fd415eef0 0x7f5fd415b620 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc005fd0 tx=0x7f5fbc02a670 comp rx=0 tx=0).stop 2024-09-10T20:58:25.088 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.083+0000 7f5fd895d640 1 -- 172.21.15.190:0/4120017559 shutdown_connections 2024-09-10T20:58:25.088 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.083+0000 7f5fd895d640 1 --2- 172.21.15.190:0/4120017559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fd415eef0 0x7f5fd415b620 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.088 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.083+0000 7f5fd895d640 1 -- 172.21.15.190:0/4120017559 >> 172.21.15.190:0/4120017559 conn(0x7f5fd40b2770 msgr2=0x7f5fd40b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:25.088 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.083+0000 7f5fd895d640 1 -- 172.21.15.190:0/4120017559 shutdown_connections 2024-09-10T20:58:25.088 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.083+0000 7f5fd895d640 1 -- 172.21.15.190:0/4120017559 wait complete. 2024-09-10T20:58:25.089 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.085+0000 7f5fd895d640 1 Processor -- start 2024-09-10T20:58:25.089 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.085+0000 7f5fd895d640 1 -- start start 2024-09-10T20:58:25.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.085+0000 7f5fd895d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fd4073650 0x7f5fd40b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:25.091 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.085+0000 7f5fd895d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fd415eef0 0x7f5fd4149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:25.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.085+0000 7f5fd895d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fd40b0c00 0x7f5fd41496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:25.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.085+0000 7f5fd895d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd40b4180 con 0x7f5fd415eef0 2024-09-10T20:58:25.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.085+0000 7f5fd895d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd4070a80 con 0x7f5fd4073650 2024-09-10T20:58:25.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.085+0000 7f5fd895d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5fd40b4300 con 0x7f5fd40b0c00 2024-09-10T20:58:25.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.085+0000 7f5fd259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fd4073650 0x7f5fd40b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:25.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fd259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fd4073650 0x7f5fd40b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:54822/0 (socket says 172.21.15.190:54822) 2024-09-10T20:58:25.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fd2d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fd40b0c00 0x7f5fd41496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:25.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fd259c640 1 -- 172.21.15.190:0/1778443015 learned_addr learned my addr 172.21.15.190:0/1778443015 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-10T20:58:25.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fcbfff640 1 --2- 172.21.15.190:0/1778443015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fd415eef0 0x7f5fd4149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:25.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fd2d9d640 1 -- 172.21.15.190:0/1778443015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fd4073650 msgr2=0x7f5fd40b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fd2d9d640 1 --2- 172.21.15.190:0/1778443015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fd4073650 0x7f5fd40b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fd2d9d640 1 -- 172.21.15.190:0/1778443015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fd415eef0 msgr2=0x7f5fd4149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.093 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fd2d9d640 1 --2- 172.21.15.190:0/1778443015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fd415eef0 0x7f5fd4149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.093 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fd2d9d640 1 -- 172.21.15.190:0/1778443015 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd40b10a0 con 0x7f5fd40b0c00 2024-09-10T20:58:25.093 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fd259c640 1 --2- 172.21.15.190:0/1778443015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fd4073650 0x7f5fd40b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:58:25.093 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fd2d9d640 1 --2- 172.21.15.190:0/1778443015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fd40b0c00 0x7f5fd41496a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40744e0 tx=0x7f5fcc00cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:25.093 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fcaffd640 1 -- 172.21.15.190:0/1778443015 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc003b70 con 0x7f5fd40b0c00 2024-09-10T20:58:25.093 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fcaffd640 1 -- 172.21.15.190:0/1778443015 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fcc003d10 con 0x7f5fd40b0c00 2024-09-10T20:58:25.093 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fd895d640 1 -- 172.21.15.190:0/1778443015 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd40b1380 con 0x7f5fd40b0c00 2024-09-10T20:58:25.093 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fcaffd640 1 -- 172.21.15.190:0/1778443015 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc005330 con 0x7f5fd40b0c00 2024-09-10T20:58:25.094 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fd895d640 1 -- 172.21.15.190:0/1778443015 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd4135050 con 0x7f5fd40b0c00 2024-09-10T20:58:25.094 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fcaffd640 1 -- 172.21.15.190:0/1778443015 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f5fcc00a640 con 0x7f5fd40b0c00 2024-09-10T20:58:25.094 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.086+0000 7f5fcaffd640 1 -- 172.21.15.190:0/1778443015 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f5fcc012d30 con 0x7f5fd40b0c00 2024-09-10T20:58:25.094 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.087+0000 7f5fd895d640 1 -- 172.21.15.190:0/1778443015 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd40b3fa0 con 0x7f5fd40b0c00 2024-09-10T20:58:25.095 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.091+0000 7f5fcaffd640 1 -- 172.21.15.190:0/1778443015 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f5fd40b3fa0 con 0x7f5fd40b0c00 2024-09-10T20:58:25.165 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.162+0000 7f5fd895d640 1 -- 172.21.15.190:0/1778443015 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "914f0013-7767-489d-aeaf-44f67aeb6810", "id": 5} v 0) -- 0x7f5fd40d0850 con 0x7f5fd40b0c00 2024-09-10T20:58:25.169 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.166+0000 7f5fcaffd640 1 -- 172.21.15.190:0/1778443015 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "914f0013-7767-489d-aeaf-44f67aeb6810", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f5fcc0056e0 con 0x7f5fd40b0c00 2024-09-10T20:58:25.169 INFO:teuthology.orchestra.run.smithi190.stdout:5 2024-09-10T20:58:25.171 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.166+0000 7f5fd895d640 1 -- 172.21.15.190:0/1778443015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fd40b0c00 msgr2=0x7f5fd41496a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.171 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.166+0000 7f5fd895d640 1 --2- 172.21.15.190:0/1778443015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fd40b0c00 0x7f5fd41496a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40744e0 tx=0x7f5fcc00cb90 comp rx=0 tx=0).stop 2024-09-10T20:58:25.171 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.166+0000 7f5fd895d640 1 -- 172.21.15.190:0/1778443015 shutdown_connections 2024-09-10T20:58:25.171 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.166+0000 7f5fd895d640 1 --2- 172.21.15.190:0/1778443015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fd40b0c00 0x7f5fd41496a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.171 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.166+0000 7f5fd895d640 1 --2- 172.21.15.190:0/1778443015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fd415eef0 0x7f5fd4149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.171 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.166+0000 7f5fd895d640 1 --2- 172.21.15.190:0/1778443015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fd4073650 0x7f5fd40b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.171 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.166+0000 7f5fd895d640 1 -- 172.21.15.190:0/1778443015 >> 172.21.15.190:0/1778443015 conn(0x7f5fd40b2770 msgr2=0x7f5fd4148630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:25.171 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.167+0000 7f5fd895d640 1 -- 172.21.15.190:0/1778443015 shutdown_connections 2024-09-10T20:58:25.171 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.167+0000 7f5fd895d640 1 -- 172.21.15.190:0/1778443015 wait complete. 2024-09-10T20:58:25.181 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb 6 2024-09-10T20:58:25.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.278+0000 7f3a0a857640 1 -- 172.21.15.190:0/1309271372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a0415dea0 msgr2=0x7f3a040d18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.278+0000 7f3a0a857640 1 --2- 172.21.15.190:0/1309271372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a0415dea0 0x7f3a040d18f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009830 tx=0x7f39f402a480 comp rx=0 tx=0).stop 2024-09-10T20:58:25.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.278+0000 7f3a0a857640 1 -- 172.21.15.190:0/1309271372 shutdown_connections 2024-09-10T20:58:25.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.278+0000 7f3a0a857640 1 --2- 172.21.15.190:0/1309271372 >> v2:172.21.15.98:3300/0 conn(0x7f3a040d2a10 0x7f3a040b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.278+0000 7f3a0a857640 1 --2- 172.21.15.190:0/1309271372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a0415dea0 0x7f3a040d18f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.278+0000 7f3a0a857640 1 -- 172.21.15.190:0/1309271372 >> 172.21.15.190:0/1309271372 conn(0x7f3a040d3db0 msgr2=0x7f3a040d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:25.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.278+0000 7f3a0a857640 1 -- 172.21.15.190:0/1309271372 shutdown_connections 2024-09-10T20:58:25.283 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.278+0000 7f3a0a857640 1 -- 172.21.15.190:0/1309271372 wait complete. 2024-09-10T20:58:25.286 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.280+0000 7f3a0a857640 1 Processor -- start 2024-09-10T20:58:25.286 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.280+0000 7f3a0a857640 1 -- start start 2024-09-10T20:58:25.286 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.280+0000 7f3a0a857640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a040d1e30 0x7f3a041595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:25.286 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.280+0000 7f3a0a857640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a040d2a10 0x7f3a0415b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:25.286 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.280+0000 7f3a0a857640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a0415dea0 0x7f3a0415b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:25.286 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.280+0000 7f3a0a857640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a040b2950 con 0x7f3a040d1e30 2024-09-10T20:58:25.286 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.280+0000 7f3a0a857640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3a040b2ad0 con 0x7f3a040d2a10 2024-09-10T20:58:25.286 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.280+0000 7f3a0a857640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3a040b2c50 con 0x7f3a0415dea0 2024-09-10T20:58:25.286 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a08df4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a0415dea0 0x7f3a0415b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:25.286 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a037fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a040d2a10 0x7f3a0415b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:25.286 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a08df4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a0415dea0 0x7f3a0415b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:43928/0 (socket says 172.21.15.190:43928) 2024-09-10T20:58:25.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a08df4640 1 -- 172.21.15.190:0/1915065723 learned_addr learned my addr 172.21.15.190:0/1915065723 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-10T20:58:25.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a03fff640 1 --2- 172.21.15.190:0/1915065723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a040d1e30 0x7f3a041595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:25.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a08df4640 1 -- 172.21.15.190:0/1915065723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a040d2a10 msgr2=0x7f3a0415b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a08df4640 1 --2- 172.21.15.190:0/1915065723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a040d2a10 0x7f3a0415b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a08df4640 1 -- 172.21.15.190:0/1915065723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a040d1e30 msgr2=0x7f3a041595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a08df4640 1 --2- 172.21.15.190:0/1915065723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a040d1e30 0x7f3a041595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a08df4640 1 -- 172.21.15.190:0/1915065723 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a040c88b0 con 0x7f3a0415dea0 2024-09-10T20:58:25.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a037fe640 1 --2- 172.21.15.190:0/1915065723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a040d2a10 0x7f3a0415b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:58:25.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a08df4640 1 --2- 172.21.15.190:0/1915065723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a0415dea0 0x7f3a0415b820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00daa0 tx=0x7f39fc00df60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:25.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a00ff9640 1 -- 172.21.15.190:0/1915065723 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc017070 con 0x7f3a0415dea0 2024-09-10T20:58:25.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a00ff9640 1 -- 172.21.15.190:0/1915065723 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39fc003970 con 0x7f3a0415dea0 2024-09-10T20:58:25.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a0a857640 1 -- 172.21.15.190:0/1915065723 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a040c95a0 con 0x7f3a0415dea0 2024-09-10T20:58:25.287 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a00ff9640 1 -- 172.21.15.190:0/1915065723 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc003dc0 con 0x7f3a0415dea0 2024-09-10T20:58:25.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.281+0000 7f3a0a857640 1 -- 172.21.15.190:0/1915065723 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a040b3600 con 0x7f3a0415dea0 2024-09-10T20:58:25.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.282+0000 7f3a00ff9640 1 -- 172.21.15.190:0/1915065723 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f39fc003f60 con 0x7f3a0415dea0 2024-09-10T20:58:25.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.282+0000 7f3a00ff9640 1 -- 172.21.15.190:0/1915065723 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f39fc01b400 con 0x7f3a0415dea0 2024-09-10T20:58:25.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.282+0000 7f3a0a857640 1 -- 172.21.15.190:0/1915065723 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a040cabb0 con 0x7f3a0415dea0 2024-09-10T20:58:25.288 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.284+0000 7f3a00ff9640 1 -- 172.21.15.190:0/1915065723 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f39fc010460 con 0x7f3a0415dea0 2024-09-10T20:58:25.360 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.356+0000 7f3a0a857640 1 -- 172.21.15.190:0/1915065723 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb", "id": 6} v 0) -- 0x7f3a040d02e0 con 0x7f3a0415dea0 2024-09-10T20:58:25.363 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.359+0000 7f3a00ff9640 1 -- 172.21.15.190:0/1915065723 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f39fc004b20 con 0x7f3a0415dea0 2024-09-10T20:58:25.363 INFO:teuthology.orchestra.run.smithi190.stdout:6 2024-09-10T20:58:25.364 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.360+0000 7f3a0a857640 1 -- 172.21.15.190:0/1915065723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a0415dea0 msgr2=0x7f3a0415b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.364 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.360+0000 7f3a0a857640 1 --2- 172.21.15.190:0/1915065723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a0415dea0 0x7f3a0415b820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00daa0 tx=0x7f39fc00df60 comp rx=0 tx=0).stop 2024-09-10T20:58:25.364 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.360+0000 7f3a0a857640 1 -- 172.21.15.190:0/1915065723 shutdown_connections 2024-09-10T20:58:25.364 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.360+0000 7f3a0a857640 1 --2- 172.21.15.190:0/1915065723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a0415dea0 0x7f3a0415b820 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.364 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.360+0000 7f3a0a857640 1 --2- 172.21.15.190:0/1915065723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a040d2a10 0x7f3a0415b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.365 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.360+0000 7f3a0a857640 1 --2- 172.21.15.190:0/1915065723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a040d1e30 0x7f3a041595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.365 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.360+0000 7f3a0a857640 1 -- 172.21.15.190:0/1915065723 >> 172.21.15.190:0/1915065723 conn(0x7f3a040d3db0 msgr2=0x7f3a0415bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:25.365 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.360+0000 7f3a0a857640 1 -- 172.21.15.190:0/1915065723 shutdown_connections 2024-09-10T20:58:25.365 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.360+0000 7f3a0a857640 1 -- 172.21.15.190:0/1915065723 wait complete. 2024-09-10T20:58:25.374 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new a4086531-b341-45bb-a07b-a03577a4a0bf 7 2024-09-10T20:58:25.475 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.471+0000 7f98769ad640 1 -- 172.21.15.190:0/4274171348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f987015e470 msgr2=0x7f987006ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.475 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.471+0000 7f98769ad640 1 --2- 172.21.15.190:0/4274171348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f987015e470 0x7f987006ca50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9860009830 tx=0x7f986002a3f0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.475 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.471+0000 7f98769ad640 1 -- 172.21.15.190:0/4274171348 shutdown_connections 2024-09-10T20:58:25.476 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.471+0000 7f98769ad640 1 --2- 172.21.15.190:0/4274171348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98700cd2e0 0x7f98700ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.476 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.471+0000 7f98769ad640 1 --2- 172.21.15.190:0/4274171348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f987015e470 0x7f987006ca50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.476 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.471+0000 7f98769ad640 1 -- 172.21.15.190:0/4274171348 >> 172.21.15.190:0/4274171348 conn(0x7f9870074170 msgr2=0x7f9870074580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:25.476 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.471+0000 7f98769ad640 1 -- 172.21.15.190:0/4274171348 shutdown_connections 2024-09-10T20:58:25.476 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.471+0000 7f98769ad640 1 -- 172.21.15.190:0/4274171348 wait complete. 2024-09-10T20:58:25.476 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.472+0000 7f98769ad640 1 Processor -- start 2024-09-10T20:58:25.477 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.472+0000 7f98769ad640 1 -- start start 2024-09-10T20:58:25.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.473+0000 7f98769ad640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9870074c90 0x7f98700b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:25.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.473+0000 7f98769ad640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98700cd2e0 0x7f98700c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:25.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.473+0000 7f98769ad640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f987015e470 0x7f98700c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:25.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.473+0000 7f98769ad640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98700b56c0 con 0x7f9870074c90 2024-09-10T20:58:25.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.473+0000 7f98769ad640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9870069590 con 0x7f987015e470 2024-09-10T20:58:25.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.473+0000 7f98769ad640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f98700b5840 con 0x7f98700cd2e0 2024-09-10T20:58:25.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.473+0000 7f9874f4a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f987015e470 0x7f98700c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:25.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.473+0000 7f986f7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98700cd2e0 0x7f98700c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:25.479 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.473+0000 7f9874f4a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f987015e470 0x7f98700c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:54842/0 (socket says 172.21.15.190:54842) 2024-09-10T20:58:25.480 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.473+0000 7f9874f4a640 1 -- 172.21.15.190:0/548689639 learned_addr learned my addr 172.21.15.190:0/548689639 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-09-10T20:58:25.480 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.473+0000 7f986ffff640 1 --2- 172.21.15.190:0/548689639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9870074c90 0x7f98700b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:25.480 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.474+0000 7f9874f4a640 1 -- 172.21.15.190:0/548689639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98700cd2e0 msgr2=0x7f98700c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.480 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.474+0000 7f9874f4a640 1 --2- 172.21.15.190:0/548689639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98700cd2e0 0x7f98700c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.480 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.474+0000 7f9874f4a640 1 -- 172.21.15.190:0/548689639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9870074c90 msgr2=0x7f98700b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.480 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.474+0000 7f9874f4a640 1 --2- 172.21.15.190:0/548689639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9870074c90 0x7f98700b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.480 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.474+0000 7f9874f4a640 1 -- 172.21.15.190:0/548689639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98701f5ae0 con 0x7f987015e470 2024-09-10T20:58:25.480 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.474+0000 7f986f7fe640 1 --2- 172.21.15.190:0/548689639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98700cd2e0 0x7f98700c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:58:25.480 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.474+0000 7f9874f4a640 1 --2- 172.21.15.190:0/548689639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f987015e470 0x7f98700c5f70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f986800efc0 tx=0x7f986800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:25.480 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.474+0000 7f986cff9640 1 -- 172.21.15.190:0/548689639 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868003970 con 0x7f987015e470 2024-09-10T20:58:25.480 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.474+0000 7f986cff9640 1 -- 172.21.15.190:0/548689639 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9868003c20 con 0x7f987015e470 2024-09-10T20:58:25.480 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.474+0000 7f98769ad640 1 -- 172.21.15.190:0/548689639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98701f5dc0 con 0x7f987015e470 2024-09-10T20:58:25.481 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.474+0000 7f986cff9640 1 -- 172.21.15.190:0/548689639 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868004280 con 0x7f987015e470 2024-09-10T20:58:25.481 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.474+0000 7f98769ad640 1 -- 172.21.15.190:0/548689639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98701f6380 con 0x7f987015e470 2024-09-10T20:58:25.481 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.474+0000 7f986cff9640 1 -- 172.21.15.190:0/548689639 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f986800b040 con 0x7f987015e470 2024-09-10T20:58:25.481 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.474+0000 7f986cff9640 1 -- 172.21.15.190:0/548689639 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f986800bc90 con 0x7f987015e470 2024-09-10T20:58:25.481 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.475+0000 7f98769ad640 1 -- 172.21.15.190:0/548689639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98700ca9a0 con 0x7f987015e470 2024-09-10T20:58:25.481 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.477+0000 7f986cff9640 1 -- 172.21.15.190:0/548689639 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f9868024070 con 0x7f987015e470 2024-09-10T20:58:25.550 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.546+0000 7f98769ad640 1 -- 172.21.15.190:0/548689639 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a4086531-b341-45bb-a07b-a03577a4a0bf", "id": 7} v 0) -- 0x7f987006caf0 con 0x7f987015e470 2024-09-10T20:58:25.553 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.549+0000 7f986cff9640 1 -- 172.21.15.190:0/548689639 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a4086531-b341-45bb-a07b-a03577a4a0bf", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f9868004420 con 0x7f987015e470 2024-09-10T20:58:25.553 INFO:teuthology.orchestra.run.smithi190.stdout:7 2024-09-10T20:58:25.554 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.550+0000 7f98769ad640 1 -- 172.21.15.190:0/548689639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f987015e470 msgr2=0x7f98700c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.554 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.550+0000 7f98769ad640 1 --2- 172.21.15.190:0/548689639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f987015e470 0x7f98700c5f70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f986800efc0 tx=0x7f986800c800 comp rx=0 tx=0).stop 2024-09-10T20:58:25.554 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.550+0000 7f98769ad640 1 -- 172.21.15.190:0/548689639 shutdown_connections 2024-09-10T20:58:25.554 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.550+0000 7f98769ad640 1 --2- 172.21.15.190:0/548689639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f987015e470 0x7f98700c5f70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.554 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.550+0000 7f98769ad640 1 --2- 172.21.15.190:0/548689639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98700cd2e0 0x7f98700c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.555 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.550+0000 7f98769ad640 1 --2- 172.21.15.190:0/548689639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9870074c90 0x7f98700b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.555 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.550+0000 7f98769ad640 1 -- 172.21.15.190:0/548689639 >> 172.21.15.190:0/548689639 conn(0x7f9870074170 msgr2=0x7f98700b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:25.555 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.550+0000 7f98769ad640 1 -- 172.21.15.190:0/548689639 shutdown_connections 2024-09-10T20:58:25.555 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:58:25.550+0000 7f98769ad640 1 -- 172.21.15.190:0/548689639 wait complete. 2024-09-10T20:58:25.564 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T20:58:25.565 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T20:58:25.567 INFO:tasks.ceph.osd.0:Started 2024-09-10T20:58:25.567 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T20:58:25.567 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T20:58:25.569 INFO:tasks.ceph.osd.1:Started 2024-09-10T20:58:25.570 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T20:58:25.570 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T20:58:25.572 INFO:tasks.ceph.osd.2:Started 2024-09-10T20:58:25.572 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-10T20:58:25.572 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-10T20:58:25.574 INFO:tasks.ceph.osd.3:Started 2024-09-10T20:58:25.574 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-10T20:58:25.574 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-10T20:58:25.576 INFO:tasks.ceph.osd.4:Started 2024-09-10T20:58:25.577 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-10T20:58:25.577 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-10T20:58:25.578 INFO:tasks.ceph.osd.5:Started 2024-09-10T20:58:25.579 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-10T20:58:25.579 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-10T20:58:25.581 INFO:tasks.ceph.osd.6:Started 2024-09-10T20:58:25.581 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-10T20:58:25.581 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-10T20:58:25.583 INFO:tasks.ceph.osd.7:Started 2024-09-10T20:58:25.583 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:58:25.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.713+0000 7f48f20b0640 1 -- 172.21.15.98:0/844152104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48ec0af010 msgr2=0x7f48ec073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.713+0000 7f48f20b0640 1 --2- 172.21.15.98:0/844152104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48ec0af010 0x7f48ec073650 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48d4005fd0 tx=0x7f48d402a5d0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.713+0000 7f48f20b0640 1 -- 172.21.15.98:0/844152104 shutdown_connections 2024-09-10T20:58:25.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.713+0000 7f48f20b0640 1 --2- 172.21.15.98:0/844152104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48ec0af010 0x7f48ec073650 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.713+0000 7f48f20b0640 1 --2- 172.21.15.98:0/844152104 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48ec151bd0 0x7f48ec0af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.713+0000 7f48f20b0640 1 -- 172.21.15.98:0/844152104 >> 172.21.15.98:0/844152104 conn(0x7f48ec0b2980 msgr2=0x7f48ec0b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:25.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.713+0000 7f48f20b0640 1 -- 172.21.15.98:0/844152104 shutdown_connections 2024-09-10T20:58:25.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.713+0000 7f48f20b0640 1 -- 172.21.15.98:0/844152104 wait complete. 2024-09-10T20:58:25.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.714+0000 7f48f20b0640 1 Processor -- start 2024-09-10T20:58:25.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.715+0000 7f48f20b0640 1 -- start start 2024-09-10T20:58:25.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.715+0000 7f48f20b0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48ec073b90 0x7f48ec153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:25.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.715+0000 7f48f20b0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48ec0af010 0x7f48ec155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:25.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.715+0000 7f48f20b0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48ec151bd0 0x7f48ec1f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:25.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.715+0000 7f48f20b0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec0ae470 con 0x7f48ec0af010 2024-09-10T20:58:25.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.715+0000 7f48f20b0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec0712f0 con 0x7f48ec073b90 2024-09-10T20:58:25.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.715+0000 7f48f20b0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f48ec0ae5f0 con 0x7f48ec151bd0 2024-09-10T20:58:25.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.715+0000 7f48e2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48ec0af010 0x7f48ec155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:25.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.715+0000 7f48e2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48ec0af010 0x7f48ec155680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49872/0 (socket says 172.21.15.98:49872) 2024-09-10T20:58:25.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.715+0000 7f48e2ffd640 1 -- 172.21.15.98:0/4275432214 learned_addr learned my addr 172.21.15.98:0/4275432214 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:25.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.715+0000 7f48ebfff640 1 --2- 172.21.15.98:0/4275432214 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48ec151bd0 0x7f48ec1f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:25.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.715+0000 7f48eb7fe640 1 --2- 172.21.15.98:0/4275432214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48ec073b90 0x7f48ec153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:25.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.716+0000 7f48e2ffd640 1 -- 172.21.15.98:0/4275432214 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48ec151bd0 msgr2=0x7f48ec1f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.716+0000 7f48e2ffd640 1 --2- 172.21.15.98:0/4275432214 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48ec151bd0 0x7f48ec1f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.716+0000 7f48e2ffd640 1 -- 172.21.15.98:0/4275432214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48ec073b90 msgr2=0x7f48ec153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.716+0000 7f48e2ffd640 1 --2- 172.21.15.98:0/4275432214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48ec073b90 0x7f48ec153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.716+0000 7f48e2ffd640 1 -- 172.21.15.98:0/4275432214 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec156020 con 0x7f48ec0af010 2024-09-10T20:58:25.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.716+0000 7f48e2ffd640 1 --2- 172.21.15.98:0/4275432214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48ec0af010 0x7f48ec155680 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009850 tx=0x7f48d4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:25.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.716+0000 7f48e8ff9640 1 -- 172.21.15.98:0/4275432214 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4035070 con 0x7f48ec0af010 2024-09-10T20:58:25.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.716+0000 7f48e8ff9640 1 -- 172.21.15.98:0/4275432214 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48d4005600 con 0x7f48ec0af010 2024-09-10T20:58:25.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.716+0000 7f48f20b0640 1 -- 172.21.15.98:0/4275432214 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ec156300 con 0x7f48ec0af010 2024-09-10T20:58:25.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.716+0000 7f48e8ff9640 1 -- 172.21.15.98:0/4275432214 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4003e60 con 0x7f48ec0af010 2024-09-10T20:58:25.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.716+0000 7f48f20b0640 1 -- 172.21.15.98:0/4275432214 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ec1568c0 con 0x7f48ec0af010 2024-09-10T20:58:25.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.716+0000 7f48e8ff9640 1 -- 172.21.15.98:0/4275432214 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f48d4038800 con 0x7f48ec0af010 2024-09-10T20:58:25.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.716+0000 7f48e8ff9640 1 -- 172.21.15.98:0/4275432214 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f48d4042af0 con 0x7f48ec0af010 2024-09-10T20:58:25.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.717+0000 7f48f20b0640 1 -- 172.21.15.98:0/4275432214 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ec0aeb10 con 0x7f48ec0af010 2024-09-10T20:58:25.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.719+0000 7f48e8ff9640 1 -- 172.21.15.98:0/4275432214 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f48d4062950 con 0x7f48ec0af010 2024-09-10T20:58:25.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.793+0000 7f48f20b0640 1 -- 172.21.15.98:0/4275432214 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48ec0d1630 con 0x7f48ec0af010 2024-09-10T20:58:25.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.794+0000 7f48e8ff9640 1 -- 172.21.15.98:0/4275432214 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f48d4050020 con 0x7f48ec0af010 2024-09-10T20:58:25.798 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:58:25.798 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":10,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:58:25.547705+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:58:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.795+0000 7f48f20b0640 1 -- 172.21.15.98:0/4275432214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48ec0af010 msgr2=0x7f48ec155680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.795+0000 7f48f20b0640 1 --2- 172.21.15.98:0/4275432214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48ec0af010 0x7f48ec155680 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009850 tx=0x7f48d4003820 comp rx=0 tx=0).stop 2024-09-10T20:58:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.795+0000 7f48f20b0640 1 -- 172.21.15.98:0/4275432214 shutdown_connections 2024-09-10T20:58:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.795+0000 7f48f20b0640 1 --2- 172.21.15.98:0/4275432214 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48ec151bd0 0x7f48ec1f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.795+0000 7f48f20b0640 1 --2- 172.21.15.98:0/4275432214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48ec0af010 0x7f48ec155680 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.795+0000 7f48f20b0640 1 --2- 172.21.15.98:0/4275432214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48ec073b90 0x7f48ec153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.795+0000 7f48f20b0640 1 -- 172.21.15.98:0/4275432214 >> 172.21.15.98:0/4275432214 conn(0x7f48ec0b2980 msgr2=0x7f48ec15a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.795+0000 7f48f20b0640 1 -- 172.21.15.98:0/4275432214 shutdown_connections 2024-09-10T20:58:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:25.795+0000 7f48f20b0640 1 -- 172.21.15.98:0/4275432214 wait complete. 2024-09-10T20:58:25.809 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T20:58:25.809 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T20:58:26.616 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T20:58:26.709 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.704+0000 7fd87d117640 1 -- 172.21.15.98:0/3635657739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8780d03b0 msgr2=0x7fd8780b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:26.709 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.704+0000 7fd87d117640 1 --2- 172.21.15.98:0/3635657739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8780d03b0 0x7fd8780b1180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd878071920 tx=0x7fd87002aa00 comp rx=0 tx=0).stop 2024-09-10T20:58:26.709 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.705+0000 7fd87d117640 1 -- 172.21.15.98:0/3635657739 shutdown_connections 2024-09-10T20:58:26.709 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.705+0000 7fd87d117640 1 --2- 172.21.15.98:0/3635657739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8780d03b0 0x7fd8780b1180 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:26.709 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.705+0000 7fd87d117640 1 --2- 172.21.15.98:0/3635657739 >> v2:172.21.15.190:3300/0 conn(0x7fd8780cffd0 0x7fd878071c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:26.709 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.705+0000 7fd87d117640 1 -- 172.21.15.98:0/3635657739 >> 172.21.15.98:0/3635657739 conn(0x7fd8780b3880 msgr2=0x7fd8780b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:26.709 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.705+0000 7fd87d117640 1 -- 172.21.15.98:0/3635657739 shutdown_connections 2024-09-10T20:58:26.709 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.705+0000 7fd87d117640 1 -- 172.21.15.98:0/3635657739 wait complete. 2024-09-10T20:58:26.710 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.706+0000 7fd87d117640 1 Processor -- start 2024-09-10T20:58:26.710 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.706+0000 7fd87d117640 1 -- start start 2024-09-10T20:58:26.712 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87d117640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8780b0c30 0x7fd8781f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:26.712 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87d117640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8780cffd0 0x7fd8780b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:26.713 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87d117640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8780d03b0 0x7fd8780cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:26.713 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87d117640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd8780cd2c0 con 0x7fd8780d03b0 2024-09-10T20:58:26.713 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87d117640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd8780cd140 con 0x7fd8780cffd0 2024-09-10T20:58:26.713 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87d117640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd8780cd440 con 0x7fd8780b0c30 2024-09-10T20:58:26.713 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8780d03b0 0x7fd8780cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:26.713 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd876d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8780b0c30 0x7fd8781f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:26.713 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87659c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8780cffd0 0x7fd8780b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:26.713 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8780d03b0 0x7fd8780cac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49892/0 (socket says 172.21.15.98:49892) 2024-09-10T20:58:26.713 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd876d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8780b0c30 0x7fd8781f34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:45236/0 (socket says 172.21.15.98:45236) 2024-09-10T20:58:26.713 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87759e640 1 -- 172.21.15.98:0/1479979285 learned_addr learned my addr 172.21.15.98:0/1479979285 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:26.713 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87759e640 1 -- 172.21.15.98:0/1479979285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8780b0c30 msgr2=0x7fd8781f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:26.713 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87759e640 1 --2- 172.21.15.98:0/1479979285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8780b0c30 0x7fd8781f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:26.714 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87759e640 1 -- 172.21.15.98:0/1479979285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8780cffd0 msgr2=0x7fd8780b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:26.714 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87759e640 1 --2- 172.21.15.98:0/1479979285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8780cffd0 0x7fd8780b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:26.714 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87759e640 1 -- 172.21.15.98:0/1479979285 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8781f39e0 con 0x7fd8780d03b0 2024-09-10T20:58:26.714 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.707+0000 7fd87759e640 1 --2- 172.21.15.98:0/1479979285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8780d03b0 0x7fd8780cac20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd870005fd0 tx=0x7fd870003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:26.714 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.708+0000 7fd85f7fe640 1 -- 172.21.15.98:0/1479979285 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870033050 con 0x7fd8780d03b0 2024-09-10T20:58:26.714 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.708+0000 7fd85f7fe640 1 -- 172.21.15.98:0/1479979285 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd870004450 con 0x7fd8780d03b0 2024-09-10T20:58:26.714 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.708+0000 7fd87d117640 1 -- 172.21.15.98:0/1479979285 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8780b7650 con 0x7fd8780d03b0 2024-09-10T20:58:26.714 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.708+0000 7fd85f7fe640 1 -- 172.21.15.98:0/1479979285 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870038450 con 0x7fd8780d03b0 2024-09-10T20:58:26.714 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.708+0000 7fd87d117640 1 -- 172.21.15.98:0/1479979285 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8780b4dd0 con 0x7fd8780d03b0 2024-09-10T20:58:26.714 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.708+0000 7fd85f7fe640 1 -- 172.21.15.98:0/1479979285 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fd870040070 con 0x7fd8780d03b0 2024-09-10T20:58:26.714 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.708+0000 7fd85f7fe640 1 -- 172.21.15.98:0/1479979285 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd87004bb10 con 0x7fd8780d03b0 2024-09-10T20:58:26.714 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.709+0000 7fd87d117640 1 -- 172.21.15.98:0/1479979285 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8780b4020 con 0x7fd8780d03b0 2024-09-10T20:58:26.715 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.710+0000 7fd85f7fe640 1 -- 172.21.15.98:0/1479979285 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fd870062950 con 0x7fd8780d03b0 2024-09-10T20:58:26.771 INFO:tasks.ceph.osd.1.smithi098.stderr:2024-09-10T20:58:26.767+0000 7f8d84016640 -1 Falling back to public interface 2024-09-10T20:58:26.774 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.770+0000 7fd87d117640 1 -- 172.21.15.98:0/1479979285 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd878071c30 con 0x7fd8780d03b0 2024-09-10T20:58:26.775 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.771+0000 7fd85f7fe640 1 -- 172.21.15.98:0/1479979285 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd870050030 con 0x7fd8780d03b0 2024-09-10T20:58:26.775 INFO:teuthology.misc.health.smithi098.stdout: 2024-09-10T20:58:26.775 INFO:teuthology.misc.health.smithi098.stdout:{"epoch":10,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:58:25.547705+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:58:26.776 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.772+0000 7fd87d117640 1 -- 172.21.15.98:0/1479979285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8780d03b0 msgr2=0x7fd8780cac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:26.776 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.772+0000 7fd87d117640 1 --2- 172.21.15.98:0/1479979285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8780d03b0 0x7fd8780cac20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd870005fd0 tx=0x7fd870003f10 comp rx=0 tx=0).stop 2024-09-10T20:58:26.776 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.772+0000 7fd87d117640 1 -- 172.21.15.98:0/1479979285 shutdown_connections 2024-09-10T20:58:26.776 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.772+0000 7fd87d117640 1 --2- 172.21.15.98:0/1479979285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8780d03b0 0x7fd8780cac20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:26.776 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.772+0000 7fd87d117640 1 --2- 172.21.15.98:0/1479979285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8780cffd0 0x7fd8780b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:26.776 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.772+0000 7fd87d117640 1 --2- 172.21.15.98:0/1479979285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8780b0c30 0x7fd8781f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:26.776 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.772+0000 7fd87d117640 1 -- 172.21.15.98:0/1479979285 >> 172.21.15.98:0/1479979285 conn(0x7fd8780b3880 msgr2=0x7fd8780b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:26.776 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.772+0000 7fd87d117640 1 -- 172.21.15.98:0/1479979285 shutdown_connections 2024-09-10T20:58:26.776 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:26.772+0000 7fd87d117640 1 -- 172.21.15.98:0/1479979285 wait complete. 2024-09-10T20:58:26.785 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T20:58:27.017 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-10T20:58:27.013+0000 7fcce8418640 -1 Falling back to public interface 2024-09-10T20:58:27.030 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-10T20:58:27.026+0000 7f2941415640 -1 Falling back to public interface 2024-09-10T20:58:27.053 INFO:tasks.ceph.osd.2.smithi098.stderr:2024-09-10T20:58:27.049+0000 7f82bf589640 -1 Falling back to public interface 2024-09-10T20:58:27.068 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-10T20:58:27.064+0000 7f3ee35d9640 -1 Falling back to public interface 2024-09-10T20:58:27.284 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-10T20:58:27.278+0000 7fd5f95bd640 -1 Falling back to public interface 2024-09-10T20:58:27.291 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-10T20:58:27.287+0000 7fb2d4d51640 -1 Falling back to public interface 2024-09-10T20:58:27.295 INFO:tasks.ceph.osd.0.smithi098.stderr:2024-09-10T20:58:27.291+0000 7fcd089fd640 -1 Falling back to public interface 2024-09-10T20:58:27.914 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-09-10T20:58:27.910+0000 7f94c5e17640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T20:58:27.914 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-09-10T20:58:27.910+0000 7f94c5e17640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T20:58:30.614 INFO:tasks.ceph.osd.1.smithi098.stderr:2024-09-10T20:58:30.610+0000 7f8d84016640 -1 osd.1 0 log_to_monitors true 2024-09-10T20:58:30.658 INFO:tasks.ceph.osd.2.smithi098.stderr:2024-09-10T20:58:30.654+0000 7f82bf589640 -1 osd.2 0 log_to_monitors true 2024-09-10T20:58:30.799 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-10T20:58:30.795+0000 7fcce8418640 -1 osd.5 0 log_to_monitors true 2024-09-10T20:58:31.061 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-10T20:58:31.057+0000 7fb2d4d51640 -1 osd.3 0 log_to_monitors true 2024-09-10T20:58:31.344 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-10T20:58:31.340+0000 7fd5f95bd640 -1 osd.4 0 log_to_monitors true 2024-09-10T20:58:31.366 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-10T20:58:31.362+0000 7f2941415640 -1 osd.6 0 log_to_monitors true 2024-09-10T20:58:31.384 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-10T20:58:31.379+0000 7f3ee35d9640 -1 osd.7 0 log_to_monitors true 2024-09-10T20:58:31.593 INFO:tasks.ceph.osd.0.smithi098.stderr:2024-09-10T20:58:31.589+0000 7fcd089fd640 -1 osd.0 0 log_to_monitors true 2024-09-10T20:58:32.692 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-10T20:58:32.688+0000 7fcce3e53640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T20:58:32.704 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-09-10T20:58:32.700+0000 7fccdbc72640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:58:33.302 INFO:tasks.ceph.osd.1.smithi098.stderr:2024-09-10T20:58:33.298+0000 7f8d801ae640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T20:58:33.317 INFO:tasks.ceph.osd.2.smithi098.stderr:2024-09-10T20:58:33.313+0000 7f82bb518640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T20:58:33.448 INFO:tasks.ceph.osd.1.smithi098.stderr:2024-09-10T20:58:33.444+0000 7f8d777b9640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:58:33.462 INFO:tasks.ceph.osd.2.smithi098.stderr:2024-09-10T20:58:33.458+0000 7f82b3337640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:58:33.591 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T20:58:33.693 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.689+0000 7fcd10d3d640 1 -- 172.21.15.98:0/1631162838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd0c06f690 msgr2=0x7fcd0c06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:33.693 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.689+0000 7fcd10d3d640 1 --2- 172.21.15.98:0/1631162838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd0c06f690 0x7fcd0c06fb40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcd04005fd0 tx=0x7fcd0402a840 comp rx=0 tx=0).stop 2024-09-10T20:58:33.693 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.689+0000 7fcd10d3d640 1 -- 172.21.15.98:0/1631162838 shutdown_connections 2024-09-10T20:58:33.693 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.689+0000 7fcd10d3d640 1 --2- 172.21.15.98:0/1631162838 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd0c06f690 0x7fcd0c06fb40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:33.694 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.689+0000 7fcd10d3d640 1 --2- 172.21.15.98:0/1631162838 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd0c071840 0x7fcd0c071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:33.694 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.689+0000 7fcd10d3d640 1 -- 172.21.15.98:0/1631162838 >> 172.21.15.98:0/1631162838 conn(0x7fcd0c0d0bb0 msgr2=0x7fcd0c0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:33.694 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.689+0000 7fcd10d3d640 1 -- 172.21.15.98:0/1631162838 shutdown_connections 2024-09-10T20:58:33.694 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.689+0000 7fcd10d3d640 1 -- 172.21.15.98:0/1631162838 wait complete. 2024-09-10T20:58:33.694 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.690+0000 7fcd10d3d640 1 Processor -- start 2024-09-10T20:58:33.694 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.690+0000 7fcd10d3d640 1 -- start start 2024-09-10T20:58:33.696 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.691+0000 7fcd10d3d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c06f690 0x7fcd0c15b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:33.696 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.691+0000 7fcd10d3d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd0c071840 0x7fcd0c0afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:33.696 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.691+0000 7fcd10d3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd0c15adb0 0x7fcd0c1496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:33.697 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.691+0000 7fcd10d3d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c0b3ff0 con 0x7fcd0c06f690 2024-09-10T20:58:33.697 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.691+0000 7fcd10d3d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c0b3e70 con 0x7fcd0c15adb0 2024-09-10T20:58:33.697 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.691+0000 7fcd10d3d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcd0c0b4170 con 0x7fcd0c071840 2024-09-10T20:58:33.697 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.691+0000 7fcd0a59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c06f690 0x7fcd0c15b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:33.697 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.691+0000 7fcd0a59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c06f690 0x7fcd0c15b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41510/0 (socket says 172.21.15.98:41510) 2024-09-10T20:58:33.697 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.691+0000 7fcd0a59c640 1 -- 172.21.15.98:0/388866945 learned_addr learned my addr 172.21.15.98:0/388866945 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:33.697 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.691+0000 7fcd03fff640 1 --2- 172.21.15.98:0/388866945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd0c071840 0x7fcd0c0afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:33.697 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.691+0000 7fcd0ad9d640 1 --2- 172.21.15.98:0/388866945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd0c15adb0 0x7fcd0c1496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:33.697 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.692+0000 7fcd0a59c640 1 -- 172.21.15.98:0/388866945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd0c071840 msgr2=0x7fcd0c0afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:33.697 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.692+0000 7fcd0a59c640 1 --2- 172.21.15.98:0/388866945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd0c071840 0x7fcd0c0afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:33.697 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.692+0000 7fcd0a59c640 1 -- 172.21.15.98:0/388866945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd0c15adb0 msgr2=0x7fcd0c1496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:33.698 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.692+0000 7fcd0a59c640 1 --2- 172.21.15.98:0/388866945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd0c15adb0 0x7fcd0c1496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:33.698 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.692+0000 7fcd0a59c640 1 -- 172.21.15.98:0/388866945 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0c149ea0 con 0x7fcd0c06f690 2024-09-10T20:58:33.698 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.692+0000 7fcd0a59c640 1 --2- 172.21.15.98:0/388866945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c06f690 0x7fcd0c15b7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00c530 tx=0x7fccfc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:33.698 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.692+0000 7fcd02ffd640 1 -- 172.21.15.98:0/388866945 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc016020 con 0x7fcd0c06f690 2024-09-10T20:58:33.698 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.692+0000 7fcd10d3d640 1 -- 172.21.15.98:0/388866945 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0c14a180 con 0x7fcd0c06f690 2024-09-10T20:58:33.698 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.692+0000 7fcd02ffd640 1 -- 172.21.15.98:0/388866945 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccfc012940 con 0x7fcd0c06f690 2024-09-10T20:58:33.698 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.692+0000 7fcd10d3d640 1 -- 172.21.15.98:0/388866945 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd0c14a740 con 0x7fcd0c06f690 2024-09-10T20:58:33.698 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.692+0000 7fcd02ffd640 1 -- 172.21.15.98:0/388866945 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc012d90 con 0x7fcd0c06f690 2024-09-10T20:58:33.699 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.693+0000 7fcd10d3d640 1 -- 172.21.15.98:0/388866945 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf0005290 con 0x7fcd0c06f690 2024-09-10T20:58:33.699 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.694+0000 7fcd02ffd640 1 -- 172.21.15.98:0/388866945 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 4) ==== 100545+0+0 (secure 0 0 0) 0x7fccfc01b400 con 0x7fcd0c06f690 2024-09-10T20:58:33.699 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.695+0000 7fcd02ffd640 1 --2- 172.21.15.98:0/388866945 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fccec077ee0 0x7fccec07a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:33.699 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.695+0000 7fcd02ffd640 1 -- 172.21.15.98:0/388866945 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2511+0+0 (secure 0 0 0) 0x7fccfc099100 con 0x7fcd0c06f690 2024-09-10T20:58:33.699 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.695+0000 7fcd03fff640 1 --2- 172.21.15.98:0/388866945 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fccec077ee0 0x7fccec07a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:33.699 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.696+0000 7fcd03fff640 1 --2- 172.21.15.98:0/388866945 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fccec077ee0 0x7fccec07a3a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fccf40095f0 tx=0x7fccf4005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:33.706 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.702+0000 7fcd02ffd640 1 -- 172.21.15.98:0/388866945 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fccfc020020 con 0x7fcd0c06f690 2024-09-10T20:58:33.982 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.978+0000 7fcd10d3d640 1 -- 172.21.15.98:0/388866945 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccf0005580 con 0x7fcd0c06f690 2024-09-10T20:58:33.982 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.978+0000 7fcd02ffd640 1 -- 172.21.15.98:0/388866945 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7555 (secure 0 0 0) 0x7fccfc062310 con 0x7fcd0c06f690 2024-09-10T20:58:33.983 INFO:teuthology.misc.health.smithi098.stdout: 2024-09-10T20:58:33.983 INFO:teuthology.misc.health.smithi098.stdout:{"epoch":13,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:58:32.915456+0000","last_up_change":"2024-09-10T20:58:32.915456+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:58:33.985 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.981+0000 7fcd10d3d640 1 -- 172.21.15.98:0/388866945 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fccec077ee0 msgr2=0x7fccec07a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:33.985 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.981+0000 7fcd10d3d640 1 --2- 172.21.15.98:0/388866945 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fccec077ee0 0x7fccec07a3a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fccf40095f0 tx=0x7fccf4005f90 comp rx=0 tx=0).stop 2024-09-10T20:58:33.985 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.981+0000 7fcd10d3d640 1 -- 172.21.15.98:0/388866945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c06f690 msgr2=0x7fcd0c15b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:33.986 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.981+0000 7fcd10d3d640 1 --2- 172.21.15.98:0/388866945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c06f690 0x7fcd0c15b7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00c530 tx=0x7fccfc00c9f0 comp rx=0 tx=0).stop 2024-09-10T20:58:33.986 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.981+0000 7fcd10d3d640 1 -- 172.21.15.98:0/388866945 shutdown_connections 2024-09-10T20:58:33.986 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.981+0000 7fcd10d3d640 1 --2- 172.21.15.98:0/388866945 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fccec077ee0 0x7fccec07a3a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:33.986 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.981+0000 7fcd10d3d640 1 --2- 172.21.15.98:0/388866945 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd0c15adb0 0x7fcd0c1496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:33.986 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.981+0000 7fcd10d3d640 1 --2- 172.21.15.98:0/388866945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd0c071840 0x7fcd0c0afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:33.986 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.981+0000 7fcd10d3d640 1 --2- 172.21.15.98:0/388866945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c06f690 0x7fcd0c15b7a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:33.986 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.981+0000 7fcd10d3d640 1 -- 172.21.15.98:0/388866945 >> 172.21.15.98:0/388866945 conn(0x7fcd0c0d0bb0 msgr2=0x7fcd0c0d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:33.986 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.981+0000 7fcd10d3d640 1 -- 172.21.15.98:0/388866945 shutdown_connections 2024-09-10T20:58:33.986 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:33.981+0000 7fcd10d3d640 1 -- 172.21.15.98:0/388866945 wait complete. 2024-09-10T20:58:33.997 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-10T20:58:34.199 INFO:tasks.ceph.osd.0.smithi098.stderr:2024-09-10T20:58:34.194+0000 7fcd0498c640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T20:58:34.209 INFO:tasks.ceph.osd.0.smithi098.stderr:2024-09-10T20:58:34.205+0000 7fccfc7ab640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:58:34.209 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-10T20:58:34.206+0000 7fb2d14f4640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T20:58:34.225 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-10T20:58:34.221+0000 7fb2c8aff640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:58:34.578 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-10T20:58:34.574+0000 7fd5f5d60640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T20:58:34.588 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-10T20:58:34.584+0000 7f3edf568640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T20:58:34.589 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-09-10T20:58:34.586+0000 7fd5ed36b640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:58:34.601 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-09-10T20:58:34.598+0000 7f3ed7387640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:58:34.617 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-10T20:58:34.613+0000 7f293ccf5640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T20:58:34.632 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-09-10T20:58:34.629+0000 7f2934b14640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:58:40.808 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T20:58:40.911 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.907+0000 7f70e4917640 1 -- 172.21.15.98:0/1179535770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e015f590 msgr2=0x7f70e00d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:40.912 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.907+0000 7f70e4917640 1 --2- 172.21.15.98:0/1179535770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e015f590 0x7f70e00d0610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70e00738a0 tx=0x7f70d802a8b0 comp rx=0 tx=0).stop 2024-09-10T20:58:40.912 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.907+0000 7f70e4917640 1 -- 172.21.15.98:0/1179535770 shutdown_connections 2024-09-10T20:58:40.912 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.907+0000 7f70e4917640 1 --2- 172.21.15.98:0/1179535770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e015f590 0x7f70e00d0610 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:40.912 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.907+0000 7f70e4917640 1 --2- 172.21.15.98:0/1179535770 >> v2:172.21.15.190:3300/0 conn(0x7f70e00cfe00 0x7f70e00cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:40.912 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.907+0000 7f70e4917640 1 -- 172.21.15.98:0/1179535770 >> 172.21.15.98:0/1179535770 conn(0x7f70e00d1db0 msgr2=0x7f70e00d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:40.912 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.907+0000 7f70e4917640 1 -- 172.21.15.98:0/1179535770 shutdown_connections 2024-09-10T20:58:40.912 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.907+0000 7f70e4917640 1 -- 172.21.15.98:0/1179535770 wait complete. 2024-09-10T20:58:40.914 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.908+0000 7f70e4917640 1 Processor -- start 2024-09-10T20:58:40.914 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.908+0000 7f70e4917640 1 -- start start 2024-09-10T20:58:40.915 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.909+0000 7f70e4917640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70e00cfe00 0x7f70e014fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:40.915 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.909+0000 7f70e4917640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70e00d01e0 0x7f70e00c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:40.915 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.909+0000 7f70e4917640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e015f590 0x7f70e006dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:40.915 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.909+0000 7f70e4917640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f70e00b1e40 con 0x7f70e015f590 2024-09-10T20:58:40.915 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.909+0000 7f70e4917640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f70e00b1cc0 con 0x7f70e00cfe00 2024-09-10T20:58:40.915 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.909+0000 7f70e4917640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f70e00b1fc0 con 0x7f70e00d01e0 2024-09-10T20:58:40.915 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.909+0000 7f70ded9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e015f590 0x7f70e006dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:40.915 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.909+0000 7f70ded9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e015f590 0x7f70e006dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60712/0 (socket says 172.21.15.98:60712) 2024-09-10T20:58:40.915 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.909+0000 7f70ded9d640 1 -- 172.21.15.98:0/4242060575 learned_addr learned my addr 172.21.15.98:0/4242060575 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:40.915 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.909+0000 7f70de59c640 1 --2- 172.21.15.98:0/4242060575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70e00cfe00 0x7f70e014fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:40.915 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.909+0000 7f70d7fff640 1 --2- 172.21.15.98:0/4242060575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70e00d01e0 0x7f70e00c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:40.915 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.910+0000 7f70ded9d640 1 -- 172.21.15.98:0/4242060575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70e00d01e0 msgr2=0x7f70e00c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:40.915 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.910+0000 7f70ded9d640 1 --2- 172.21.15.98:0/4242060575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70e00d01e0 0x7f70e00c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:40.916 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.910+0000 7f70ded9d640 1 -- 172.21.15.98:0/4242060575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70e00cfe00 msgr2=0x7f70e014fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:40.916 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.910+0000 7f70ded9d640 1 --2- 172.21.15.98:0/4242060575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70e00cfe00 0x7f70e014fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:40.916 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.910+0000 7f70ded9d640 1 -- 172.21.15.98:0/4242060575 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e0153530 con 0x7f70e015f590 2024-09-10T20:58:40.916 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.910+0000 7f70ded9d640 1 --2- 172.21.15.98:0/4242060575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e015f590 0x7f70e006dc10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009ca0 tx=0x7f70d8003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:40.916 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.910+0000 7f70d6ffd640 1 -- 172.21.15.98:0/4242060575 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d8031990 con 0x7f70e015f590 2024-09-10T20:58:40.916 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.910+0000 7f70d6ffd640 1 -- 172.21.15.98:0/4242060575 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70d8031c20 con 0x7f70e015f590 2024-09-10T20:58:40.916 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.910+0000 7f70e4917640 1 -- 172.21.15.98:0/4242060575 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e00c79a0 con 0x7f70e015f590 2024-09-10T20:58:40.916 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.910+0000 7f70d6ffd640 1 -- 172.21.15.98:0/4242060575 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d80330d0 con 0x7f70e015f590 2024-09-10T20:58:40.916 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.910+0000 7f70e4917640 1 -- 172.21.15.98:0/4242060575 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e00c7670 con 0x7f70e015f590 2024-09-10T20:58:40.917 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.911+0000 7f70e4917640 1 -- 172.21.15.98:0/4242060575 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e00cd160 con 0x7f70e015f590 2024-09-10T20:58:40.917 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.912+0000 7f70d6ffd640 1 -- 172.21.15.98:0/4242060575 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f70d8033290 con 0x7f70e015f590 2024-09-10T20:58:40.917 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.912+0000 7f70d6ffd640 1 --2- 172.21.15.98:0/4242060575 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f70c0077f70 0x7f70c007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:40.918 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.912+0000 7f70d6ffd640 1 -- 172.21.15.98:0/4242060575 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f70d80bad20 con 0x7f70e015f590 2024-09-10T20:58:40.918 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.912+0000 7f70de59c640 1 --2- 172.21.15.98:0/4242060575 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f70c0077f70 0x7f70c007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:40.918 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.913+0000 7f70de59c640 1 --2- 172.21.15.98:0/4242060575 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f70c0077f70 0x7f70c007a430 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f70e006fdf0 tx=0x7f70d000b390 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:40.920 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:40.917+0000 7f70d6ffd640 1 -- 172.21.15.98:0/4242060575 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f70d803f030 con 0x7f70e015f590 2024-09-10T20:58:41.098 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:41.094+0000 7f70e4917640 1 -- 172.21.15.98:0/4242060575 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70e00715c0 con 0x7f70e015f590 2024-09-10T20:58:41.098 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:41.095+0000 7f70d6ffd640 1 -- 172.21.15.98:0/4242060575 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12908 (secure 0 0 0) 0x7f70d80835b0 con 0x7f70e015f590 2024-09-10T20:58:41.099 INFO:teuthology.misc.health.smithi098.stdout: 2024-09-10T20:58:41.099 INFO:teuthology.misc.health.smithi098.stdout:{"epoch":17,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:58:37.593850+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:58:41.101 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:41.097+0000 7f70e4917640 1 -- 172.21.15.98:0/4242060575 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f70c0077f70 msgr2=0x7f70c007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:41.101 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:41.097+0000 7f70e4917640 1 --2- 172.21.15.98:0/4242060575 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f70c0077f70 0x7f70c007a430 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f70e006fdf0 tx=0x7f70d000b390 comp rx=0 tx=0).stop 2024-09-10T20:58:41.101 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:41.097+0000 7f70e4917640 1 -- 172.21.15.98:0/4242060575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e015f590 msgr2=0x7f70e006dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:41.101 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:41.097+0000 7f70e4917640 1 --2- 172.21.15.98:0/4242060575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e015f590 0x7f70e006dc10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009ca0 tx=0x7f70d8003990 comp rx=0 tx=0).stop 2024-09-10T20:58:41.101 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:41.097+0000 7f70e4917640 1 -- 172.21.15.98:0/4242060575 shutdown_connections 2024-09-10T20:58:41.101 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:41.097+0000 7f70e4917640 1 --2- 172.21.15.98:0/4242060575 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f70c0077f70 0x7f70c007a430 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:41.101 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:41.097+0000 7f70e4917640 1 --2- 172.21.15.98:0/4242060575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70e015f590 0x7f70e006dc10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:41.102 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:41.097+0000 7f70e4917640 1 --2- 172.21.15.98:0/4242060575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70e00d01e0 0x7f70e00c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:41.102 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:41.097+0000 7f70e4917640 1 --2- 172.21.15.98:0/4242060575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70e00cfe00 0x7f70e014fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:41.102 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:41.097+0000 7f70e4917640 1 -- 172.21.15.98:0/4242060575 >> 172.21.15.98:0/4242060575 conn(0x7f70e00d1db0 msgr2=0x7f70e00d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:41.102 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:41.097+0000 7f70e4917640 1 -- 172.21.15.98:0/4242060575 shutdown_connections 2024-09-10T20:58:41.102 INFO:teuthology.misc.health.smithi098.stderr:2024-09-10T20:58:41.097+0000 7f70e4917640 1 -- 172.21.15.98:0/4242060575 wait complete. 2024-09-10T20:58:41.110 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-10T20:58:41.111 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T20:58:41.111 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T20:58:41.111 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T20:58:41.114 INFO:tasks.ceph.mds.a:Started 2024-09-10T20:58:41.114 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T20:58:41.114 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T20:58:41.116 INFO:tasks.ceph.mds.c:Started 2024-09-10T20:58:41.116 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T20:58:41.117 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T20:58:41.119 INFO:tasks.ceph.mds.b:Started 2024-09-10T20:58:41.119 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T20:58:41.120 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T20:58:41.122 INFO:tasks.ceph.mds.d:Started 2024-09-10T20:58:41.122 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-10T20:58:41.122 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:58:41.188 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-10T20:58:41.189 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-10T20:58:41.205 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2024-09-10T20:58:41.205 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2024-09-10T20:58:41.205 INFO:tasks.ceph.mds.d.smithi190.stderr:2024-09-10T20:58:41.202+0000 7f94c466f640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-10T20:58:41.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.223+0000 7f4fccd82640 1 -- 172.21.15.98:0/1987073794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc80b4250 msgr2=0x7f4fc80b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:41.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.223+0000 7f4fccd82640 1 --2- 172.21.15.98:0/1987073794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc80b4250 0x7f4fc80b4700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0005fd0 tx=0x7f4fc002a8a0 comp rx=0 tx=0).stop 2024-09-10T20:58:41.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.224+0000 7f4fccd82640 1 -- 172.21.15.98:0/1987073794 shutdown_connections 2024-09-10T20:58:41.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.224+0000 7f4fccd82640 1 --2- 172.21.15.98:0/1987073794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc80b4250 0x7f4fc80b4700 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:41.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.224+0000 7f4fccd82640 1 --2- 172.21.15.98:0/1987073794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc80d08b0 0x7f4fc80d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:41.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.224+0000 7f4fccd82640 1 -- 172.21.15.98:0/1987073794 >> 172.21.15.98:0/1987073794 conn(0x7f4fc809d190 msgr2=0x7f4fc80d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:41.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.224+0000 7f4fccd82640 1 -- 172.21.15.98:0/1987073794 shutdown_connections 2024-09-10T20:58:41.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.224+0000 7f4fccd82640 1 -- 172.21.15.98:0/1987073794 wait complete. 2024-09-10T20:58:41.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.225+0000 7f4fccd82640 1 Processor -- start 2024-09-10T20:58:41.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.225+0000 7f4fccd82640 1 -- start start 2024-09-10T20:58:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fccd82640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc80b4250 0x7f4fc80cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fccd82640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc80cec40 0x7f4fc80b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fccd82640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc80d08b0 0x7f4fc814f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fccd82640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc80b4fd0 con 0x7f4fc80d08b0 2024-09-10T20:58:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fccd82640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc80b4e50 con 0x7f4fc80b4250 2024-09-10T20:58:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fccd82640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc80b5150 con 0x7f4fc80cec40 2024-09-10T20:58:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fc6d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc80d08b0 0x7f4fc814f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fc6d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc80d08b0 0x7f4fc814f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60740/0 (socket says 172.21.15.98:60740) 2024-09-10T20:58:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fc6d9d640 1 -- 172.21.15.98:0/3928749271 learned_addr learned my addr 172.21.15.98:0/3928749271 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fc659c640 1 --2- 172.21.15.98:0/3928749271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc80b4250 0x7f4fc80cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fbdd9b640 1 --2- 172.21.15.98:0/3928749271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc80cec40 0x7f4fc80b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fc6d9d640 1 -- 172.21.15.98:0/3928749271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc80cec40 msgr2=0x7f4fc80b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fc6d9d640 1 --2- 172.21.15.98:0/3928749271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc80cec40 0x7f4fc80b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fc6d9d640 1 -- 172.21.15.98:0/3928749271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc80b4250 msgr2=0x7f4fc80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fc6d9d640 1 --2- 172.21.15.98:0/3928749271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc80b4250 0x7f4fc80cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.226+0000 7f4fc6d9d640 1 -- 172.21.15.98:0/3928749271 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc80b7e10 con 0x7f4fc80d08b0 2024-09-10T20:58:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.227+0000 7f4fc6d9d640 1 --2- 172.21.15.98:0/3928749271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc80d08b0 0x7f4fc814f280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009cd0 tx=0x7f4fc0032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.227+0000 7f4fbf7fe640 1 -- 172.21.15.98:0/3928749271 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0031430 con 0x7f4fc80d08b0 2024-09-10T20:58:41.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.227+0000 7f4fbf7fe640 1 -- 172.21.15.98:0/3928749271 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fc0004470 con 0x7f4fc80d08b0 2024-09-10T20:58:41.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.227+0000 7f4fbf7fe640 1 -- 172.21.15.98:0/3928749271 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0045410 con 0x7f4fc80d08b0 2024-09-10T20:58:41.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.227+0000 7f4fccd82640 1 -- 172.21.15.98:0/3928749271 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc80b80f0 con 0x7f4fc80d08b0 2024-09-10T20:58:41.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.227+0000 7f4fccd82640 1 -- 172.21.15.98:0/3928749271 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc80b86b0 con 0x7f4fc80d08b0 2024-09-10T20:58:41.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.228+0000 7f4fccd82640 1 -- 172.21.15.98:0/3928749271 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc80b4700 con 0x7f4fc80d08b0 2024-09-10T20:58:41.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.228+0000 7f4fbf7fe640 1 -- 172.21.15.98:0/3928749271 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4fc0030070 con 0x7f4fc80d08b0 2024-09-10T20:58:41.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.229+0000 7f4fbf7fe640 1 --2- 172.21.15.98:0/3928749271 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4fa8077f70 0x7f4fa807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:41.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.229+0000 7f4fbf7fe640 1 -- 172.21.15.98:0/3928749271 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f4fc002e020 con 0x7f4fc80d08b0 2024-09-10T20:58:41.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.229+0000 7f4fc659c640 1 --2- 172.21.15.98:0/3928749271 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4fa8077f70 0x7f4fa807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:41.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.229+0000 7f4fc659c640 1 --2- 172.21.15.98:0/3928749271 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4fa8077f70 0x7f4fa807a430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80095f0 tx=0x7f4fb8005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:41.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.233+0000 7f4fbf7fe640 1 -- 172.21.15.98:0/3928749271 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4fc0083220 con 0x7f4fc80d08b0 2024-09-10T20:58:41.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.442+0000 7f4fccd82640 1 -- 172.21.15.98:0/3928749271 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fc80d0690 con 0x7f4fc80d08b0 2024-09-10T20:58:41.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.442+0000 7f4fbf7fe640 1 -- 172.21.15.98:0/3928749271 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7f4fc0087c50 con 0x7f4fc80d08b0 2024-09-10T20:58:41.447 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:58:41.447 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":2,"btime":"2024-09-10T20:58:41:189425+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1984550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1984550306},{"type":"v1","addr":"172.21.15.98:6833","nonce":1984550306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T20:58:41.447 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 2 2024-09-10T20:58:41.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.444+0000 7f4fccd82640 1 -- 172.21.15.98:0/3928749271 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4fa8077f70 msgr2=0x7f4fa807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:41.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.444+0000 7f4fccd82640 1 --2- 172.21.15.98:0/3928749271 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4fa8077f70 0x7f4fa807a430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80095f0 tx=0x7f4fb8005f90 comp rx=0 tx=0).stop 2024-09-10T20:58:41.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.444+0000 7f4fccd82640 1 -- 172.21.15.98:0/3928749271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc80d08b0 msgr2=0x7f4fc814f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:41.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.444+0000 7f4fccd82640 1 --2- 172.21.15.98:0/3928749271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc80d08b0 0x7f4fc814f280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009cd0 tx=0x7f4fc0032d30 comp rx=0 tx=0).stop 2024-09-10T20:58:41.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.445+0000 7f4fccd82640 1 -- 172.21.15.98:0/3928749271 shutdown_connections 2024-09-10T20:58:41.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.445+0000 7f4fccd82640 1 --2- 172.21.15.98:0/3928749271 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4fa8077f70 0x7f4fa807a430 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:41.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.445+0000 7f4fccd82640 1 --2- 172.21.15.98:0/3928749271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc80d08b0 0x7f4fc814f280 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:41.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.445+0000 7f4fccd82640 1 --2- 172.21.15.98:0/3928749271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc80cec40 0x7f4fc80b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:41.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.445+0000 7f4fccd82640 1 --2- 172.21.15.98:0/3928749271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc80b4250 0x7f4fc80cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:41.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.445+0000 7f4fccd82640 1 -- 172.21.15.98:0/3928749271 >> 172.21.15.98:0/3928749271 conn(0x7f4fc809d190 msgr2=0x7f4fc80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:41.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.445+0000 7f4fccd82640 1 -- 172.21.15.98:0/3928749271 shutdown_connections 2024-09-10T20:58:41.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:41.445+0000 7f4fccd82640 1 -- 172.21.15.98:0/3928749271 wait complete. 2024-09-10T20:58:43.460 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:58:43.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.568+0000 7fd6a522a640 1 -- 172.21.15.98:0/3105351968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a015e060 msgr2=0x7fd6a00b6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:43.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.568+0000 7fd6a522a640 1 --2- 172.21.15.98:0/3105351968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a015e060 0x7fd6a00b6dc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd698009830 tx=0x7fd69802a510 comp rx=0 tx=0).stop 2024-09-10T20:58:43.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.568+0000 7fd6a522a640 1 -- 172.21.15.98:0/3105351968 shutdown_connections 2024-09-10T20:58:43.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.568+0000 7fd6a522a640 1 --2- 172.21.15.98:0/3105351968 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6a00d4af0 0x7fd6a00cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:43.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.568+0000 7fd6a522a640 1 --2- 172.21.15.98:0/3105351968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a015e060 0x7fd6a00b6dc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:43.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.568+0000 7fd6a522a640 1 -- 172.21.15.98:0/3105351968 >> 172.21.15.98:0/3105351968 conn(0x7fd6a00d3f70 msgr2=0x7fd6a00d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:43.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.568+0000 7fd6a522a640 1 -- 172.21.15.98:0/3105351968 shutdown_connections 2024-09-10T20:58:43.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.568+0000 7fd6a522a640 1 -- 172.21.15.98:0/3105351968 wait complete. 2024-09-10T20:58:43.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.570+0000 7fd6a522a640 1 Processor -- start 2024-09-10T20:58:43.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.570+0000 7fd6a522a640 1 -- start start 2024-09-10T20:58:43.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd6a522a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6a00d4af0 0x7fd6a0153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:43.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd6a522a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a015e060 0x7fd6a01f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:43.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd6a522a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6a00cf680 0x7fd6a01f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:43.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd6a522a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a00cef80 con 0x7fd6a015e060 2024-09-10T20:58:43.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd6a522a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a00d2eb0 con 0x7fd6a00cf680 2024-09-10T20:58:43.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd6a522a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a00cf100 con 0x7fd6a00d4af0 2024-09-10T20:58:43.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd69e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a015e060 0x7fd6a01f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:43.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd69e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a015e060 0x7fd6a01f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60750/0 (socket says 172.21.15.98:60750) 2024-09-10T20:58:43.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd69e59c640 1 -- 172.21.15.98:0/2667940586 learned_addr learned my addr 172.21.15.98:0/2667940586 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:43.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd69ed9d640 1 --2- 172.21.15.98:0/2667940586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6a00d4af0 0x7fd6a0153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:43.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd69f59e640 1 --2- 172.21.15.98:0/2667940586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6a00cf680 0x7fd6a01f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:43.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd69e59c640 1 -- 172.21.15.98:0/2667940586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6a00d4af0 msgr2=0x7fd6a0153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:43.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd69e59c640 1 --2- 172.21.15.98:0/2667940586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6a00d4af0 0x7fd6a0153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:43.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd69e59c640 1 -- 172.21.15.98:0/2667940586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6a00cf680 msgr2=0x7fd6a01f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:43.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd69e59c640 1 --2- 172.21.15.98:0/2667940586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6a00cf680 0x7fd6a01f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:43.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd69e59c640 1 -- 172.21.15.98:0/2667940586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a00ca3c0 con 0x7fd6a015e060 2024-09-10T20:58:43.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd69e59c640 1 --2- 172.21.15.98:0/2667940586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a015e060 0x7fd6a01f4310 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c500 tx=0x7fd68800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:43.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.571+0000 7fd6977fe640 1 -- 172.21.15.98:0/2667940586 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688017070 con 0x7fd6a015e060 2024-09-10T20:58:43.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.572+0000 7fd6977fe640 1 -- 172.21.15.98:0/2667940586 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6880035f0 con 0x7fd6a015e060 2024-09-10T20:58:43.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.572+0000 7fd6a522a640 1 -- 172.21.15.98:0/2667940586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a00ca6a0 con 0x7fd6a015e060 2024-09-10T20:58:43.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.572+0000 7fd6977fe640 1 -- 172.21.15.98:0/2667940586 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688003a40 con 0x7fd6a015e060 2024-09-10T20:58:43.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.572+0000 7fd6a522a640 1 -- 172.21.15.98:0/2667940586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a00cd5f0 con 0x7fd6a015e060 2024-09-10T20:58:43.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.573+0000 7fd6a522a640 1 -- 172.21.15.98:0/2667940586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a00cea40 con 0x7fd6a015e060 2024-09-10T20:58:43.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.573+0000 7fd6977fe640 1 -- 172.21.15.98:0/2667940586 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd688003be0 con 0x7fd6a015e060 2024-09-10T20:58:43.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.574+0000 7fd6977fe640 1 --2- 172.21.15.98:0/2667940586 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd680077fa0 0x7fd68007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:43.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.574+0000 7fd6977fe640 1 -- 172.21.15.98:0/2667940586 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd688099100 con 0x7fd6a015e060 2024-09-10T20:58:43.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.574+0000 7fd69ed9d640 1 --2- 172.21.15.98:0/2667940586 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd680077fa0 0x7fd68007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:43.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.574+0000 7fd69ed9d640 1 --2- 172.21.15.98:0/2667940586 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd680077fa0 0x7fd68007a460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd698034000 tx=0x7fd698004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:43.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.578+0000 7fd6977fe640 1 -- 172.21.15.98:0/2667940586 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd688061fe0 con 0x7fd6a015e060 2024-09-10T20:58:43.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.785+0000 7fd6a522a640 1 -- 172.21.15.98:0/2667940586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6a008dfa0 con 0x7fd6a015e060 2024-09-10T20:58:43.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.786+0000 7fd6977fe640 1 -- 172.21.15.98:0/2667940586 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3903 (secure 0 0 0) 0x7fd6880618e0 con 0x7fd6a015e060 2024-09-10T20:58:43.790 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:58:43.790 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":3,"btime":"2024-09-10T20:58:42:190434+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4277,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/1954552281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1954552281},{"type":"v1","addr":"172.21.15.190:6836","nonce":1954552281}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/8997965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":8997965},{"type":"v1","addr":"172.21.15.190:6837","nonce":8997965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1984550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1984550306},{"type":"v1","addr":"172.21.15.98:6833","nonce":1984550306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4380,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3141703317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3141703317},{"type":"v1","addr":"172.21.15.98:6835","nonce":3141703317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T20:58:43.790 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 3 2024-09-10T20:58:43.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.788+0000 7fd6a522a640 1 -- 172.21.15.98:0/2667940586 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd680077fa0 msgr2=0x7fd68007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:43.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.788+0000 7fd6a522a640 1 --2- 172.21.15.98:0/2667940586 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd680077fa0 0x7fd68007a460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd698034000 tx=0x7fd698004730 comp rx=0 tx=0).stop 2024-09-10T20:58:43.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.788+0000 7fd6a522a640 1 -- 172.21.15.98:0/2667940586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a015e060 msgr2=0x7fd6a01f4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:43.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.788+0000 7fd6a522a640 1 --2- 172.21.15.98:0/2667940586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a015e060 0x7fd6a01f4310 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c500 tx=0x7fd68800c9c0 comp rx=0 tx=0).stop 2024-09-10T20:58:43.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.788+0000 7fd6a522a640 1 -- 172.21.15.98:0/2667940586 shutdown_connections 2024-09-10T20:58:43.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.788+0000 7fd6a522a640 1 --2- 172.21.15.98:0/2667940586 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd680077fa0 0x7fd68007a460 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:43.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.788+0000 7fd6a522a640 1 --2- 172.21.15.98:0/2667940586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6a00cf680 0x7fd6a01f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:43.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.788+0000 7fd6a522a640 1 --2- 172.21.15.98:0/2667940586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a015e060 0x7fd6a01f4310 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:43.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.788+0000 7fd6a522a640 1 --2- 172.21.15.98:0/2667940586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6a00d4af0 0x7fd6a0153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:43.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.788+0000 7fd6a522a640 1 -- 172.21.15.98:0/2667940586 >> 172.21.15.98:0/2667940586 conn(0x7fd6a00d3f70 msgr2=0x7fd6a00b7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:43.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.788+0000 7fd6a522a640 1 -- 172.21.15.98:0/2667940586 shutdown_connections 2024-09-10T20:58:43.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.788+0000 7fd6a522a640 1 -- 172.21.15.98:0/2667940586 wait complete. 2024-09-10T20:58:43.802 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T20:58:43.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.899+0000 7f5d1fe52640 1 -- 172.21.15.98:0/4075056899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d04004d90 msgr2=0x7f5d040051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:43.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.899+0000 7f5d1fe52640 1 --2- 172.21.15.98:0/4075056899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d04004d90 0x7f5d040051f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d10001dc0 tx=0x7f5d04005830 comp rx=0 tx=0).stop 2024-09-10T20:58:43.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.899+0000 7f5d1e3ef640 1 -- 172.21.15.98:0/4075056899 reap_dead start 2024-09-10T20:58:43.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.899+0000 7f5d1fe52640 1 -- 172.21.15.98:0/4075056899 shutdown_connections 2024-09-10T20:58:43.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.899+0000 7f5d1fe52640 1 -- 172.21.15.98:0/4075056899 >> 172.21.15.98:0/4075056899 conn(0x7f5d180b37c0 msgr2=0x7f5d180b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:43.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.899+0000 7f5d1fe52640 1 -- 172.21.15.98:0/4075056899 shutdown_connections 2024-09-10T20:58:43.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.899+0000 7f5d1fe52640 1 -- 172.21.15.98:0/4075056899 wait complete. 2024-09-10T20:58:43.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.901+0000 7f5d1fe52640 1 Processor -- start 2024-09-10T20:58:43.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.901+0000 7f5d1fe52640 1 -- start start 2024-09-10T20:58:43.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.901+0000 7f5d1fe52640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d04004d90 0x7f5d181f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:43.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.901+0000 7f5d1fe52640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d180b6eb0 0x7f5d180cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:43.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.901+0000 7f5d1fe52640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d180cce00 0x7f5d181f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:43.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.901+0000 7f5d1fe52640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5d180b42e0 con 0x7f5d180b6eb0 2024-09-10T20:58:43.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.901+0000 7f5d1fe52640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5d180b4460 con 0x7f5d04004d90 2024-09-10T20:58:43.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.901+0000 7f5d1fe52640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5d180b45e0 con 0x7f5d180cce00 2024-09-10T20:58:43.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.901+0000 7f5d1d3ed640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d180b6eb0 0x7f5d180cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:43.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.901+0000 7f5d1d3ed640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d180b6eb0 0x7f5d180cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60774/0 (socket says 172.21.15.98:60774) 2024-09-10T20:58:43.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.901+0000 7f5d1d3ed640 1 -- 172.21.15.98:0/2334626975 learned_addr learned my addr 172.21.15.98:0/2334626975 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:43.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.902+0000 7f5d1dbee640 1 --2- 172.21.15.98:0/2334626975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d04004d90 0x7f5d181f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:43.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.902+0000 7f5d1e3ef640 1 --2- 172.21.15.98:0/2334626975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d180cce00 0x7f5d181f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:43.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.902+0000 7f5d1d3ed640 1 -- 172.21.15.98:0/2334626975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d180cce00 msgr2=0x7f5d181f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:43.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.902+0000 7f5d1d3ed640 1 --2- 172.21.15.98:0/2334626975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d180cce00 0x7f5d181f49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:43.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.902+0000 7f5d1d3ed640 1 -- 172.21.15.98:0/2334626975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d04004d90 msgr2=0x7f5d181f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:43.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.902+0000 7f5d1d3ed640 1 --2- 172.21.15.98:0/2334626975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d04004d90 0x7f5d181f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:43.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.902+0000 7f5d1d3ed640 1 -- 172.21.15.98:0/2334626975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d180b6130 con 0x7f5d180b6eb0 2024-09-10T20:58:43.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.902+0000 7f5d1d3ed640 1 --2- 172.21.15.98:0/2334626975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d180b6eb0 0x7f5d180cc8c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800e640 tx=0x7f5d0800eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:43.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.902+0000 7f5d0e7fc640 1 -- 172.21.15.98:0/2334626975 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08015070 con 0x7f5d180b6eb0 2024-09-10T20:58:43.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.902+0000 7f5d0e7fc640 1 -- 172.21.15.98:0/2334626975 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d0800ca30 con 0x7f5d180b6eb0 2024-09-10T20:58:43.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.902+0000 7f5d1fe52640 1 -- 172.21.15.98:0/2334626975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d181f5740 con 0x7f5d180b6eb0 2024-09-10T20:58:43.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.902+0000 7f5d0e7fc640 1 -- 172.21.15.98:0/2334626975 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0800ce60 con 0x7f5d180b6eb0 2024-09-10T20:58:43.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.902+0000 7f5d1fe52640 1 -- 172.21.15.98:0/2334626975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d180b66b0 con 0x7f5d180b6eb0 2024-09-10T20:58:43.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.903+0000 7f5d1fe52640 1 -- 172.21.15.98:0/2334626975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d180d1fe0 con 0x7f5d180b6eb0 2024-09-10T20:58:43.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.903+0000 7f5d0e7fc640 1 -- 172.21.15.98:0/2334626975 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5d08011460 con 0x7f5d180b6eb0 2024-09-10T20:58:43.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.904+0000 7f5d0e7fc640 1 --2- 172.21.15.98:0/2334626975 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5cfc077f70 0x7f5cfc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:43.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.904+0000 7f5d0e7fc640 1 -- 172.21.15.98:0/2334626975 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f5d0809a0f0 con 0x7f5d180b6eb0 2024-09-10T20:58:43.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.904+0000 7f5d1dbee640 1 --2- 172.21.15.98:0/2334626975 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5cfc077f70 0x7f5cfc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:43.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.905+0000 7f5d1dbee640 1 --2- 172.21.15.98:0/2334626975 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5cfc077f70 0x7f5cfc07a430 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009b90 tx=0x7f5d14005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:43.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:43.908+0000 7f5d0e7fc640 1 -- 172.21.15.98:0/2334626975 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5d08062fd0 con 0x7f5d180b6eb0 2024-09-10T20:58:44.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.083+0000 7f5d1fe52640 1 -- 172.21.15.98:0/2334626975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5d180b7c10 con 0x7f5d180b6eb0 2024-09-10T20:58:44.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.083+0000 7f5d0e7fc640 1 -- 172.21.15.98:0/2334626975 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f5d080628d0 con 0x7f5d180b6eb0 2024-09-10T20:58:44.087 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:58:44.087 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-10T20:58:44.087 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:58:44.087 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-09-10T20:58:44.088 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-09-10T20:58:44.088 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:58:44.088 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-10T20:58:44.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.085+0000 7f5d1fe52640 1 -- 172.21.15.98:0/2334626975 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5cfc077f70 msgr2=0x7f5cfc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:44.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.085+0000 7f5d1fe52640 1 --2- 172.21.15.98:0/2334626975 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5cfc077f70 0x7f5cfc07a430 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009b90 tx=0x7f5d14005f90 comp rx=0 tx=0).stop 2024-09-10T20:58:44.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.085+0000 7f5d1fe52640 1 -- 172.21.15.98:0/2334626975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d180b6eb0 msgr2=0x7f5d180cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:44.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.085+0000 7f5d1fe52640 1 --2- 172.21.15.98:0/2334626975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d180b6eb0 0x7f5d180cc8c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800e640 tx=0x7f5d0800eb00 comp rx=0 tx=0).stop 2024-09-10T20:58:44.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.085+0000 7f5d1fe52640 1 -- 172.21.15.98:0/2334626975 shutdown_connections 2024-09-10T20:58:44.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.085+0000 7f5d1fe52640 1 --2- 172.21.15.98:0/2334626975 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5cfc077f70 0x7f5cfc07a430 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:44.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.085+0000 7f5d1fe52640 1 --2- 172.21.15.98:0/2334626975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5d180cce00 0x7f5d181f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:44.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.085+0000 7f5d1fe52640 1 --2- 172.21.15.98:0/2334626975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5d180b6eb0 0x7f5d180cc8c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:44.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.085+0000 7f5d1fe52640 1 --2- 172.21.15.98:0/2334626975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d04004d90 0x7f5d181f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:44.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.085+0000 7f5d1fe52640 1 -- 172.21.15.98:0/2334626975 >> 172.21.15.98:0/2334626975 conn(0x7f5d180b37c0 msgr2=0x7f5d180b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:44.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.085+0000 7f5d1fe52640 1 -- 172.21.15.98:0/2334626975 shutdown_connections 2024-09-10T20:58:44.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.085+0000 7f5d1fe52640 1 -- 172.21.15.98:0/2334626975 wait complete. 2024-09-10T20:58:44.099 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T20:58:44.100 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T20:58:44.126 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T20:58:44.145 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T20:58:44.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.202+0000 7f4a6ca33640 1 -- 172.21.15.98:0/2419773688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a6806cac0 msgr2=0x7f4a680ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:44.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.202+0000 7f4a6ca33640 1 --2- 172.21.15.98:0/2419773688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a6806cac0 0x7f4a680ce160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4a680740a0 tx=0x7f4a5c02a560 comp rx=0 tx=0).stop 2024-09-10T20:58:44.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.202+0000 7f4a6ca33640 1 -- 172.21.15.98:0/2419773688 shutdown_connections 2024-09-10T20:58:44.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.202+0000 7f4a6ca33640 1 --2- 172.21.15.98:0/2419773688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a6806cac0 0x7f4a680ce160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:44.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.202+0000 7f4a6ca33640 1 --2- 172.21.15.98:0/2419773688 >> v2:172.21.15.190:3300/0 conn(0x7f4a6815ae10 0x7f4a68074cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:44.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.202+0000 7f4a6ca33640 1 -- 172.21.15.98:0/2419773688 >> 172.21.15.98:0/2419773688 conn(0x7f4a68074190 msgr2=0x7f4a680745a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:44.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.202+0000 7f4a6ca33640 1 -- 172.21.15.98:0/2419773688 shutdown_connections 2024-09-10T20:58:44.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.202+0000 7f4a6ca33640 1 -- 172.21.15.98:0/2419773688 wait complete. 2024-09-10T20:58:44.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.203+0000 7f4a6ca33640 1 Processor -- start 2024-09-10T20:58:44.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.203+0000 7f4a6ca33640 1 -- start start 2024-09-10T20:58:44.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a6ca33640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a6806cac0 0x7f4a6809c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:44.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a6ca33640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a680b10a0 0x7f4a68155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:44.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a6ca33640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a6815ae10 0x7f4a6815c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:44.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a6ca33640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a680b5830 con 0x7f4a6806cac0 2024-09-10T20:58:44.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a6ca33640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4a680b56b0 con 0x7f4a6815ae10 2024-09-10T20:58:44.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a6ca33640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4a680b59b0 con 0x7f4a680b10a0 2024-09-10T20:58:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a6659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a6806cac0 0x7f4a6809c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a6659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a6806cac0 0x7f4a6809c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60798/0 (socket says 172.21.15.98:60798) 2024-09-10T20:58:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a6659c640 1 -- 172.21.15.98:0/3921786133 learned_addr learned my addr 172.21.15.98:0/3921786133 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a66d9d640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a6815ae10 0x7f4a6815c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a65d9b640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a680b10a0 0x7f4a68155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a6659c640 1 -- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a680b10a0 msgr2=0x7f4a68155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a6659c640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a680b10a0 0x7f4a68155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a6659c640 1 -- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a6815ae10 msgr2=0x7f4a6815c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a6659c640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a6815ae10 0x7f4a6815c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a6659c640 1 -- 172.21.15.98:0/3921786133 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6815d080 con 0x7f4a6806cac0 2024-09-10T20:58:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.204+0000 7f4a66d9d640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a6815ae10 0x7f4a6815c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:58:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.205+0000 7f4a6659c640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a6806cac0 0x7f4a6809c330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800eb60 tx=0x7f4a5800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.205+0000 7f4a56ffd640 1 -- 172.21.15.98:0/3921786133 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58010630 con 0x7f4a6806cac0 2024-09-10T20:58:44.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.205+0000 7f4a56ffd640 1 -- 172.21.15.98:0/3921786133 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a580108c0 con 0x7f4a6806cac0 2024-09-10T20:58:44.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.205+0000 7f4a56ffd640 1 -- 172.21.15.98:0/3921786133 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58010d10 con 0x7f4a6806cac0 2024-09-10T20:58:44.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.205+0000 7f4a6ca33640 1 -- 172.21.15.98:0/3921786133 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6815d360 con 0x7f4a6806cac0 2024-09-10T20:58:44.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.205+0000 7f4a6ca33640 1 -- 172.21.15.98:0/3921786133 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a6815d920 con 0x7f4a6806cac0 2024-09-10T20:58:44.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.206+0000 7f4a6ca33640 1 -- 172.21.15.98:0/3921786133 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a680b5f40 con 0x7f4a6806cac0 2024-09-10T20:58:44.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.206+0000 7f4a56ffd640 1 -- 172.21.15.98:0/3921786133 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4a58018420 con 0x7f4a6806cac0 2024-09-10T20:58:44.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.207+0000 7f4a56ffd640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4a48077fa0 0x7f4a4807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:44.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.207+0000 7f4a56ffd640 1 -- 172.21.15.98:0/3921786133 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f4a5809a690 con 0x7f4a6806cac0 2024-09-10T20:58:44.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.207+0000 7f4a65d9b640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4a48077fa0 0x7f4a4807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:44.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.207+0000 7f4a65d9b640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4a48077fa0 0x7f4a4807a460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c033000 tx=0x7f4a5c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:44.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.211+0000 7f4a56ffd640 1 -- 172.21.15.98:0/3921786133 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4a5800ee60 con 0x7f4a6806cac0 2024-09-10T20:58:44.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:44.386+0000 7f4a6ca33640 1 -- 172.21.15.98:0/3921786133 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4a68157dc0 con 0x7f4a6806cac0 2024-09-10T20:58:45.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.202+0000 7f4a56ffd640 1 -- 172.21.15.98:0/3921786133 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f4a58063570 con 0x7f4a6806cac0 2024-09-10T20:58:45.206 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-09-10T20:58:45.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.205+0000 7f4a6ca33640 1 -- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4a48077fa0 msgr2=0x7f4a4807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:45.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.205+0000 7f4a6ca33640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4a48077fa0 0x7f4a4807a460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c033000 tx=0x7f4a5c004730 comp rx=0 tx=0).stop 2024-09-10T20:58:45.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.205+0000 7f4a6ca33640 1 -- 172.21.15.98:0/3921786133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a6806cac0 msgr2=0x7f4a6809c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:45.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.205+0000 7f4a6ca33640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a6806cac0 0x7f4a6809c330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800eb60 tx=0x7f4a5800c6a0 comp rx=0 tx=0).stop 2024-09-10T20:58:45.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.205+0000 7f4a6ca33640 1 -- 172.21.15.98:0/3921786133 shutdown_connections 2024-09-10T20:58:45.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.205+0000 7f4a6ca33640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4a48077fa0 0x7f4a4807a460 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c033000 tx=0x7f4a5c004730 comp rx=0 tx=0).stop 2024-09-10T20:58:45.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.205+0000 7f4a6ca33640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a6815ae10 0x7f4a6815c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:45.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.205+0000 7f4a6ca33640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a680b10a0 0x7f4a68155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:45.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.205+0000 7f4a6ca33640 1 --2- 172.21.15.98:0/3921786133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a6806cac0 0x7f4a6809c330 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800eb60 tx=0x7f4a5800c6a0 comp rx=0 tx=0).stop 2024-09-10T20:58:45.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.205+0000 7f4a6ca33640 1 -- 172.21.15.98:0/3921786133 >> 172.21.15.98:0/3921786133 conn(0x7f4a68074190 msgr2=0x7f4a680b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:45.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.206+0000 7f4a6ca33640 1 -- 172.21.15.98:0/3921786133 shutdown_connections 2024-09-10T20:58:45.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.206+0000 7f4a6ca33640 1 -- 172.21.15.98:0/3921786133 wait complete. 2024-09-10T20:58:45.226 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-10T20:58:45.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.373+0000 7f47396e2640 1 -- 172.21.15.98:0/3825556295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f471c003a00 msgr2=0x7f471c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:45.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.373+0000 7f47396e2640 1 --2- 172.21.15.98:0/3825556295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f471c003a00 0x7f471c003e20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4724009ed0 tx=0x7f471c005830 comp rx=0 tx=0).stop 2024-09-10T20:58:45.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.373+0000 7f47337fe640 1 -- 172.21.15.98:0/3825556295 reap_dead start 2024-09-10T20:58:45.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.373+0000 7f47396e2640 1 -- 172.21.15.98:0/3825556295 shutdown_connections 2024-09-10T20:58:45.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.373+0000 7f47396e2640 1 -- 172.21.15.98:0/3825556295 >> 172.21.15.98:0/3825556295 conn(0x7f47340d3f70 msgr2=0x7f47340d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:45.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.373+0000 7f47396e2640 1 -- 172.21.15.98:0/3825556295 shutdown_connections 2024-09-10T20:58:45.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.373+0000 7f47396e2640 1 -- 172.21.15.98:0/3825556295 wait complete. 2024-09-10T20:58:45.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.375+0000 7f47396e2640 1 Processor -- start 2024-09-10T20:58:45.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.375+0000 7f47396e2640 1 -- start start 2024-09-10T20:58:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.375+0000 7f47396e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f471c003a00 0x7f47340cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.375+0000 7f47396e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47340ca050 0x7f47340ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.375+0000 7f47396e2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4734146400 0x7f47341468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.375+0000 7f47396e2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f47340b6dc0 con 0x7f4734146400 2024-09-10T20:58:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.375+0000 7f47396e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f47340d2e80 con 0x7f47340ca050 2024-09-10T20:58:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.375+0000 7f47396e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f47340b6f40 con 0x7f471c003a00 2024-09-10T20:58:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.375+0000 7f47337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4734146400 0x7f47341468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f47337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4734146400 0x7f47341468b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60812/0 (socket says 172.21.15.98:60812) 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f47337fe640 1 -- 172.21.15.98:0/1271088312 learned_addr learned my addr 172.21.15.98:0/1271088312 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f4732ffd640 1 --2- 172.21.15.98:0/1271088312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f471c003a00 0x7f47340cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f47327fc640 1 --2- 172.21.15.98:0/1271088312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47340ca050 0x7f47340ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f47337fe640 1 -- 172.21.15.98:0/1271088312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f471c003a00 msgr2=0x7f47340cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f47337fe640 1 --2- 172.21.15.98:0/1271088312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f471c003a00 0x7f47340cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f47337fe640 1 -- 172.21.15.98:0/1271088312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47340ca050 msgr2=0x7f47340ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f47337fe640 1 --2- 172.21.15.98:0/1271088312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47340ca050 0x7f47340ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f47337fe640 1 -- 172.21.15.98:0/1271088312 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47340cd380 con 0x7f4734146400 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f47337fe640 1 --2- 172.21.15.98:0/1271088312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4734146400 0x7f47341468b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f473406d2a0 tx=0x7f472c011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f471b7fe640 1 -- 172.21.15.98:0/1271088312 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c004440 con 0x7f4734146400 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f471b7fe640 1 -- 172.21.15.98:0/1271088312 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f472c00f8f0 con 0x7f4734146400 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f47396e2640 1 -- 172.21.15.98:0/1271088312 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47340cd650 con 0x7f4734146400 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f471b7fe640 1 -- 172.21.15.98:0/1271088312 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c00fd40 con 0x7f4734146400 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.376+0000 7f47396e2640 1 -- 172.21.15.98:0/1271088312 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47340cdc10 con 0x7f4734146400 2024-09-10T20:58:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.377+0000 7f47396e2640 1 -- 172.21.15.98:0/1271088312 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f473406aa20 con 0x7f4734146400 2024-09-10T20:58:45.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.378+0000 7f471b7fe640 1 -- 172.21.15.98:0/1271088312 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f472c018030 con 0x7f4734146400 2024-09-10T20:58:45.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.378+0000 7f471b7fe640 1 --2- 172.21.15.98:0/1271088312 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4720077f70 0x7f472007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:45.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.378+0000 7f471b7fe640 1 -- 172.21.15.98:0/1271088312 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f472c09a140 con 0x7f4734146400 2024-09-10T20:58:45.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.379+0000 7f4732ffd640 1 --2- 172.21.15.98:0/1271088312 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4720077f70 0x7f472007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:45.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.379+0000 7f4732ffd640 1 --2- 172.21.15.98:0/1271088312 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4720077f70 0x7f472007a430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4724035000 tx=0x7f472402d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:45.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.382+0000 7f471b7fe640 1 -- 172.21.15.98:0/1271088312 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f473406aa20 con 0x7f4734146400 2024-09-10T20:58:45.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:45.560+0000 7f47396e2640 1 -- 172.21.15.98:0/1271088312 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f47340cea40 con 0x7f4734146400 2024-09-10T20:58:46.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.204+0000 7f471b7fe640 1 -- 172.21.15.98:0/1271088312 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7f472c062e20 con 0x7f4734146400 2024-09-10T20:58:46.209 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-09-10T20:58:46.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.207+0000 7f47396e2640 1 -- 172.21.15.98:0/1271088312 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4720077f70 msgr2=0x7f472007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:46.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.207+0000 7f47396e2640 1 --2- 172.21.15.98:0/1271088312 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4720077f70 0x7f472007a430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4724035000 tx=0x7f472402d040 comp rx=0 tx=0).stop 2024-09-10T20:58:46.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.207+0000 7f47396e2640 1 -- 172.21.15.98:0/1271088312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4734146400 msgr2=0x7f47341468b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:46.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.207+0000 7f47396e2640 1 --2- 172.21.15.98:0/1271088312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4734146400 0x7f47341468b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f473406d2a0 tx=0x7f472c011c60 comp rx=0 tx=0).stop 2024-09-10T20:58:46.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.208+0000 7f47396e2640 1 -- 172.21.15.98:0/1271088312 shutdown_connections 2024-09-10T20:58:46.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.208+0000 7f47396e2640 1 --2- 172.21.15.98:0/1271088312 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4720077f70 0x7f472007a430 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4724035000 tx=0x7f472402d040 comp rx=0 tx=0).stop 2024-09-10T20:58:46.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.208+0000 7f47396e2640 1 --2- 172.21.15.98:0/1271088312 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4734146400 0x7f47341468b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:46.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.208+0000 7f47396e2640 1 --2- 172.21.15.98:0/1271088312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47340ca050 0x7f47340ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:46.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.208+0000 7f47396e2640 1 --2- 172.21.15.98:0/1271088312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f471c003a00 0x7f47340cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:46.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.208+0000 7f47396e2640 1 -- 172.21.15.98:0/1271088312 >> 172.21.15.98:0/1271088312 conn(0x7f47340d3f70 msgr2=0x7f4734162280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:46.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.209+0000 7f47396e2640 1 -- 172.21.15.98:0/1271088312 shutdown_connections 2024-09-10T20:58:46.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.210+0000 7f47396e2640 1 -- 172.21.15.98:0/1271088312 wait complete. 2024-09-10T20:58:46.237 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T20:58:46.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.338+0000 7f58975ec640 1 -- 172.21.15.98:0/1154867313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f589015e6d0 msgr2=0x7f589015eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:46.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.338+0000 7f58975ec640 1 --2- 172.21.15.98:0/1154867313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f589015e6d0 0x7f589015eab0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5890075110 tx=0x7f588802a560 comp rx=0 tx=0).stop 2024-09-10T20:58:46.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.338+0000 7f58975ec640 1 -- 172.21.15.98:0/1154867313 shutdown_connections 2024-09-10T20:58:46.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.338+0000 7f58975ec640 1 --2- 172.21.15.98:0/1154867313 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58900b4000 0x7f589015b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:46.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.338+0000 7f58975ec640 1 --2- 172.21.15.98:0/1154867313 >> v2:172.21.15.190:3300/0 conn(0x7f58900b3b70 0x7f589015b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:46.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.338+0000 7f58975ec640 1 --2- 172.21.15.98:0/1154867313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f589015e6d0 0x7f589015eab0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:46.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.338+0000 7f58975ec640 1 -- 172.21.15.98:0/1154867313 >> 172.21.15.98:0/1154867313 conn(0x7f58900d0c00 msgr2=0x7f58900d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:46.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.338+0000 7f58975ec640 1 -- 172.21.15.98:0/1154867313 shutdown_connections 2024-09-10T20:58:46.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.338+0000 7f58975ec640 1 -- 172.21.15.98:0/1154867313 wait complete. 2024-09-10T20:58:46.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.339+0000 7f58975ec640 1 Processor -- start 2024-09-10T20:58:46.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.339+0000 7f58975ec640 1 -- start start 2024-09-10T20:58:46.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.340+0000 7f58975ec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58900b3b70 0x7f5890147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:46.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.340+0000 7f58975ec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58900b4000 0x7f58900c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:46.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.340+0000 7f58975ec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f589015e6d0 0x7f58900c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:46.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.340+0000 7f58975ec640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f589015bce0 con 0x7f58900b3b70 2024-09-10T20:58:46.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.340+0000 7f58975ec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f589015bb60 con 0x7f589015e6d0 2024-09-10T20:58:46.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.340+0000 7f58975ec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f589015be60 con 0x7f58900b4000 2024-09-10T20:58:46.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.340+0000 7f5895388640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58900b3b70 0x7f5890147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:46.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.340+0000 7f5895388640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58900b3b70 0x7f5890147240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60832/0 (socket says 172.21.15.98:60832) 2024-09-10T20:58:46.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.340+0000 7f5895388640 1 -- 172.21.15.98:0/2907372190 learned_addr learned my addr 172.21.15.98:0/2907372190 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:46.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.341+0000 7f5895388640 1 -- 172.21.15.98:0/2907372190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58900b4000 msgr2=0x7f58900c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:46.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.341+0000 7f5895388640 1 --2- 172.21.15.98:0/2907372190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58900b4000 0x7f58900c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:46.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.341+0000 7f5895b89640 1 --2- 172.21.15.98:0/2907372190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f589015e6d0 0x7f58900c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:46.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.341+0000 7f5895388640 1 -- 172.21.15.98:0/2907372190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f589015e6d0 msgr2=0x7f58900c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:46.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.341+0000 7f5895388640 1 --2- 172.21.15.98:0/2907372190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f589015e6d0 0x7f58900c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:46.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.341+0000 7f5895388640 1 -- 172.21.15.98:0/2907372190 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5890147780 con 0x7f58900b3b70 2024-09-10T20:58:46.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.341+0000 7f5895388640 1 --2- 172.21.15.98:0/2907372190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58900b3b70 0x7f5890147240 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5888034040 tx=0x7f58880039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:46.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.341+0000 7f588dffb640 1 -- 172.21.15.98:0/2907372190 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588803e030 con 0x7f58900b3b70 2024-09-10T20:58:46.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.341+0000 7f588dffb640 1 -- 172.21.15.98:0/2907372190 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58880047a0 con 0x7f58900b3b70 2024-09-10T20:58:46.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.341+0000 7f588dffb640 1 -- 172.21.15.98:0/2907372190 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888035040 con 0x7f58900b3b70 2024-09-10T20:58:46.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.341+0000 7f58975ec640 1 -- 172.21.15.98:0/2907372190 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58900c6880 con 0x7f58900b3b70 2024-09-10T20:58:46.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.341+0000 7f58975ec640 1 -- 172.21.15.98:0/2907372190 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58901f68c0 con 0x7f58900b3b70 2024-09-10T20:58:46.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.342+0000 7f58975ec640 1 -- 172.21.15.98:0/2907372190 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f589015eab0 con 0x7f58900b3b70 2024-09-10T20:58:46.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.343+0000 7f588dffb640 1 -- 172.21.15.98:0/2907372190 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5888004bf0 con 0x7f58900b3b70 2024-09-10T20:58:46.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.343+0000 7f588dffb640 1 --2- 172.21.15.98:0/2907372190 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f587c077f70 0x7f587c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:46.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.343+0000 7f588dffb640 1 -- 172.21.15.98:0/2907372190 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f58880ba7c0 con 0x7f58900b3b70 2024-09-10T20:58:46.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.344+0000 7f5894b87640 1 --2- 172.21.15.98:0/2907372190 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f587c077f70 0x7f587c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:46.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.344+0000 7f5894b87640 1 --2- 172.21.15.98:0/2907372190 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f587c077f70 0x7f587c07a430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f589015a780 tx=0x7f5884006c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:46.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.347+0000 7f588dffb640 1 -- 172.21.15.98:0/2907372190 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5888083320 con 0x7f58900b3b70 2024-09-10T20:58:46.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:46.559+0000 7f58975ec640 1 -- 172.21.15.98:0/2907372190 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5890151c30 con 0x7f58900b3b70 2024-09-10T20:58:47.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.216+0000 7f588dffb640 1 -- 172.21.15.98:0/2907372190 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f5888082c20 con 0x7f58900b3b70 2024-09-10T20:58:47.220 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T20:58:47.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.221+0000 7f58975ec640 1 -- 172.21.15.98:0/2907372190 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f587c077f70 msgr2=0x7f587c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:47.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.221+0000 7f58975ec640 1 --2- 172.21.15.98:0/2907372190 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f587c077f70 0x7f587c07a430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f589015a780 tx=0x7f5884006c90 comp rx=0 tx=0).stop 2024-09-10T20:58:47.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.221+0000 7f58975ec640 1 -- 172.21.15.98:0/2907372190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58900b3b70 msgr2=0x7f5890147240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:47.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.221+0000 7f58975ec640 1 --2- 172.21.15.98:0/2907372190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58900b3b70 0x7f5890147240 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5888034040 tx=0x7f58880039e0 comp rx=0 tx=0).stop 2024-09-10T20:58:47.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.222+0000 7f58975ec640 1 -- 172.21.15.98:0/2907372190 shutdown_connections 2024-09-10T20:58:47.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.222+0000 7f58975ec640 1 --2- 172.21.15.98:0/2907372190 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f587c077f70 0x7f587c07a430 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:47.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.222+0000 7f58975ec640 1 --2- 172.21.15.98:0/2907372190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f589015e6d0 0x7f58900c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:47.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.222+0000 7f58975ec640 1 --2- 172.21.15.98:0/2907372190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58900b4000 0x7f58900c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:47.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.222+0000 7f58975ec640 1 --2- 172.21.15.98:0/2907372190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58900b3b70 0x7f5890147240 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:47.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.222+0000 7f58975ec640 1 -- 172.21.15.98:0/2907372190 >> 172.21.15.98:0/2907372190 conn(0x7f58900d0c00 msgr2=0x7f589015c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:47.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.222+0000 7f58975ec640 1 -- 172.21.15.98:0/2907372190 shutdown_connections 2024-09-10T20:58:47.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.222+0000 7f58975ec640 1 -- 172.21.15.98:0/2907372190 wait complete. 2024-09-10T20:58:47.254 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-10T20:58:47.255 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-10T20:58:47.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.361+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1392739035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9c00d2a70 msgr2=0x7fa9c00b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:47.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.361+0000 7fa9c58bd640 1 --2- 172.21.15.98:0/1392739035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9c00d2a70 0x7fa9c00b3a30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4009830 tx=0x7fa9b402a480 comp rx=0 tx=0).stop 2024-09-10T20:58:47.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.361+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1392739035 shutdown_connections 2024-09-10T20:58:47.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.361+0000 7fa9c58bd640 1 --2- 172.21.15.98:0/1392739035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9c00d2a70 0x7fa9c00b3a30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:47.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.361+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1392739035 >> 172.21.15.98:0/1392739035 conn(0x7fa9c009d170 msgr2=0x7fa9c00d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:47.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.361+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1392739035 shutdown_connections 2024-09-10T20:58:47.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.361+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1392739035 wait complete. 2024-09-10T20:58:47.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.362+0000 7fa9c58bd640 1 Processor -- start 2024-09-10T20:58:47.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.362+0000 7fa9c58bd640 1 -- start start 2024-09-10T20:58:47.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9c58bd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9c00b55a0 0x7fa9c009dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:47.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9c58bd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9c00d2a70 0x7fa9c01f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:47.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9c58bd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9c0161560 0x7fa9c01f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:47.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9c58bd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c00b4680 con 0x7fa9c00d2a70 2024-09-10T20:58:47.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9c58bd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c00d4830 con 0x7fa9c0161560 2024-09-10T20:58:47.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9c58bd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c00b4800 con 0x7fa9c00b55a0 2024-09-10T20:58:47.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9be7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9c00d2a70 0x7fa9c01f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:47.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9be7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9c00d2a70 0x7fa9c01f7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60844/0 (socket says 172.21.15.98:60844) 2024-09-10T20:58:47.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9be7fc640 1 -- 172.21.15.98:0/1495293191 learned_addr learned my addr 172.21.15.98:0/1495293191 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:47.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9bf7fe640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9c0161560 0x7fa9c01f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:47.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9beffd640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9c00b55a0 0x7fa9c009dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:47.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9be7fc640 1 -- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9c00b55a0 msgr2=0x7fa9c009dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:47.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9be7fc640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9c00b55a0 0x7fa9c009dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:47.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9be7fc640 1 -- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9c0161560 msgr2=0x7fa9c01f79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:47.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9be7fc640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9c0161560 0x7fa9c01f79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:47.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9be7fc640 1 -- 172.21.15.98:0/1495293191 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c015f550 con 0x7fa9c00d2a70 2024-09-10T20:58:47.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9bf7fe640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9c0161560 0x7fa9c01f79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:58:47.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9beffd640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9c00b55a0 0x7fa9c009dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:58:47.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9be7fc640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9c00d2a70 0x7fa9c01f7190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4000c00 tx=0x7fa9b4032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:47.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.363+0000 7fa9a7fff640 1 -- 172.21.15.98:0/1495293191 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4040070 con 0x7fa9c00d2a70 2024-09-10T20:58:47.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.364+0000 7fa9a7fff640 1 -- 172.21.15.98:0/1495293191 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9b4003af0 con 0x7fa9c00d2a70 2024-09-10T20:58:47.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.364+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1495293191 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c00b99a0 con 0x7fa9c00d2a70 2024-09-10T20:58:47.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.364+0000 7fa9a7fff640 1 -- 172.21.15.98:0/1495293191 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4045460 con 0x7fa9c00d2a70 2024-09-10T20:58:47.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.364+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1495293191 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c015f7f0 con 0x7fa9c00d2a70 2024-09-10T20:58:47.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.365+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1495293191 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c00caa80 con 0x7fa9c00d2a70 2024-09-10T20:58:47.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.365+0000 7fa9a7fff640 1 -- 172.21.15.98:0/1495293191 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa9b404d030 con 0x7fa9c00d2a70 2024-09-10T20:58:47.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.366+0000 7fa9a7fff640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa9ac077f70 0x7fa9ac07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:47.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.366+0000 7fa9a7fff640 1 -- 172.21.15.98:0/1495293191 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa9b40bb370 con 0x7fa9c00d2a70 2024-09-10T20:58:47.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.366+0000 7fa9beffd640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa9ac077f70 0x7fa9ac07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:47.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.366+0000 7fa9beffd640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa9ac077f70 0x7fa9ac07a430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00012e0 tx=0x7fa9b0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:47.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.370+0000 7fa9a7fff640 1 -- 172.21.15.98:0/1495293191 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa9b4083f00 con 0x7fa9c00d2a70 2024-09-10T20:58:47.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:47.556+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1495293191 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fa9c00d2310 con 0x7fa9c00d2a70 2024-09-10T20:58:48.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.218+0000 7fa9a7fff640 1 -- 172.21.15.98:0/1495293191 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7fa9b4083800 con 0x7fa9c00d2a70 2024-09-10T20:58:48.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.221+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa9ac077f70 msgr2=0x7fa9ac07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:48.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.221+0000 7fa9c58bd640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa9ac077f70 0x7fa9ac07a430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00012e0 tx=0x7fa9b0009040 comp rx=0 tx=0).stop 2024-09-10T20:58:48.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.221+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1495293191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9c00d2a70 msgr2=0x7fa9c01f7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:48.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.221+0000 7fa9c58bd640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9c00d2a70 0x7fa9c01f7190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4000c00 tx=0x7fa9b4032d50 comp rx=0 tx=0).stop 2024-09-10T20:58:48.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.221+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1495293191 shutdown_connections 2024-09-10T20:58:48.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.221+0000 7fa9c58bd640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa9ac077f70 0x7fa9ac07a430 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:48.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.221+0000 7fa9c58bd640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9c0161560 0x7fa9c01f79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:48.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.221+0000 7fa9c58bd640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9c00d2a70 0x7fa9c01f7190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:48.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.221+0000 7fa9c58bd640 1 --2- 172.21.15.98:0/1495293191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9c00b55a0 0x7fa9c009dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:48.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.221+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1495293191 >> 172.21.15.98:0/1495293191 conn(0x7fa9c009d170 msgr2=0x7fa9c00d3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:48.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.221+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1495293191 shutdown_connections 2024-09-10T20:58:48.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.221+0000 7fa9c58bd640 1 -- 172.21.15.98:0/1495293191 wait complete. 2024-09-10T20:58:48.238 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T20:58:48.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.349+0000 7fd687ae6640 1 -- 172.21.15.98:0/1508554794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd68015deb0 msgr2=0x7fd68006db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:48.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.349+0000 7fd687ae6640 1 --2- 172.21.15.98:0/1508554794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd68015deb0 0x7fd68006db60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd680069c70 tx=0x7fd67c02a5f0 comp rx=0 tx=0).stop 2024-09-10T20:58:48.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.349+0000 7fd687ae6640 1 -- 172.21.15.98:0/1508554794 shutdown_connections 2024-09-10T20:58:48.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.349+0000 7fd687ae6640 1 --2- 172.21.15.98:0/1508554794 >> v2:172.21.15.190:3300/0 conn(0x7fd68006e0a0 0x7fd68006e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:48.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.349+0000 7fd687ae6640 1 --2- 172.21.15.98:0/1508554794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd68015deb0 0x7fd68006db60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:48.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.349+0000 7fd687ae6640 1 -- 172.21.15.98:0/1508554794 >> 172.21.15.98:0/1508554794 conn(0x7fd6800d1bf0 msgr2=0x7fd6800d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:48.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.349+0000 7fd687ae6640 1 -- 172.21.15.98:0/1508554794 shutdown_connections 2024-09-10T20:58:48.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.349+0000 7fd687ae6640 1 -- 172.21.15.98:0/1508554794 wait complete. 2024-09-10T20:58:48.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.350+0000 7fd687ae6640 1 Processor -- start 2024-09-10T20:58:48.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.350+0000 7fd687ae6640 1 -- start start 2024-09-10T20:58:48.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd687ae6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd68006e0a0 0x7fd680153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd687ae6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6800b68d0 0x7fd68009b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd687ae6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd68015deb0 0x7fd6800c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd687ae6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6800d1190 con 0x7fd6800b68d0 2024-09-10T20:58:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd687ae6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd6800d1010 con 0x7fd68015deb0 2024-09-10T20:58:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd687ae6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd68015e490 con 0x7fd68006e0a0 2024-09-10T20:58:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd685081640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6800b68d0 0x7fd68009b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd685081640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6800b68d0 0x7fd68009b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60858/0 (socket says 172.21.15.98:60858) 2024-09-10T20:58:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd685081640 1 -- 172.21.15.98:0/3560573704 learned_addr learned my addr 172.21.15.98:0/3560573704 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd686083640 1 --2- 172.21.15.98:0/3560573704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd68015deb0 0x7fd6800c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd685882640 1 --2- 172.21.15.98:0/3560573704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd68006e0a0 0x7fd680153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd685081640 1 -- 172.21.15.98:0/3560573704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd68006e0a0 msgr2=0x7fd680153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd685081640 1 --2- 172.21.15.98:0/3560573704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd68006e0a0 0x7fd680153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:48.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd685081640 1 -- 172.21.15.98:0/3560573704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd68015deb0 msgr2=0x7fd6800c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:48.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd685081640 1 --2- 172.21.15.98:0/3560573704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd68015deb0 0x7fd6800c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:48.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd685081640 1 -- 172.21.15.98:0/3560573704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68009cfd0 con 0x7fd6800b68d0 2024-09-10T20:58:48.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd685081640 1 --2- 172.21.15.98:0/3560573704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6800b68d0 0x7fd68009b9d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd67000d980 tx=0x7fd67000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:48.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd6767fc640 1 -- 172.21.15.98:0/3560573704 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670017070 con 0x7fd6800b68d0 2024-09-10T20:58:48.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd6767fc640 1 -- 172.21.15.98:0/3560573704 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd670009d20 con 0x7fd6800b68d0 2024-09-10T20:58:48.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd6767fc640 1 -- 172.21.15.98:0/3560573704 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700106c0 con 0x7fd6800b68d0 2024-09-10T20:58:48.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd687ae6640 1 -- 172.21.15.98:0/3560573704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68009c340 con 0x7fd6800b68d0 2024-09-10T20:58:48.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.351+0000 7fd687ae6640 1 -- 172.21.15.98:0/3560573704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68009c900 con 0x7fd6800b68d0 2024-09-10T20:58:48.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.352+0000 7fd687ae6640 1 -- 172.21.15.98:0/3560573704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd68006e4e0 con 0x7fd6800b68d0 2024-09-10T20:58:48.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.353+0000 7fd6767fc640 1 -- 172.21.15.98:0/3560573704 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd670010860 con 0x7fd6800b68d0 2024-09-10T20:58:48.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.353+0000 7fd6767fc640 1 --2- 172.21.15.98:0/3560573704 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd664077f70 0x7fd66407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:48.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.354+0000 7fd6767fc640 1 -- 172.21.15.98:0/3560573704 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7fd67009a310 con 0x7fd6800b68d0 2024-09-10T20:58:48.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.354+0000 7fd685882640 1 --2- 172.21.15.98:0/3560573704 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd664077f70 0x7fd66407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:48.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.354+0000 7fd685882640 1 --2- 172.21.15.98:0/3560573704 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd664077f70 0x7fd66407a430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd67c033000 tx=0x7fd67c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:48.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.358+0000 7fd6767fc640 1 -- 172.21.15.98:0/3560573704 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd670062ce0 con 0x7fd6800b68d0 2024-09-10T20:58:48.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:48.533+0000 7fd687ae6640 1 -- 172.21.15.98:0/3560573704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd6800b2c40 con 0x7fd6800b68d0 2024-09-10T20:58:50.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.245+0000 7fd6767fc640 1 -- 172.21.15.98:0/3560573704 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7fd6700625e0 con 0x7fd6800b68d0 2024-09-10T20:58:50.249 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data_ec' created 2024-09-10T20:58:50.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.248+0000 7fd687ae6640 1 -- 172.21.15.98:0/3560573704 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd664077f70 msgr2=0x7fd66407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:50.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.248+0000 7fd687ae6640 1 --2- 172.21.15.98:0/3560573704 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd664077f70 0x7fd66407a430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd67c033000 tx=0x7fd67c004730 comp rx=0 tx=0).stop 2024-09-10T20:58:50.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.248+0000 7fd687ae6640 1 -- 172.21.15.98:0/3560573704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6800b68d0 msgr2=0x7fd68009b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:50.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.248+0000 7fd687ae6640 1 --2- 172.21.15.98:0/3560573704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6800b68d0 0x7fd68009b9d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd67000d980 tx=0x7fd67000de40 comp rx=0 tx=0).stop 2024-09-10T20:58:50.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.249+0000 7fd687ae6640 1 -- 172.21.15.98:0/3560573704 shutdown_connections 2024-09-10T20:58:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.249+0000 7fd687ae6640 1 --2- 172.21.15.98:0/3560573704 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd664077f70 0x7fd66407a430 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.249+0000 7fd687ae6640 1 --2- 172.21.15.98:0/3560573704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd68015deb0 0x7fd6800c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.249+0000 7fd687ae6640 1 --2- 172.21.15.98:0/3560573704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6800b68d0 0x7fd68009b9d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.249+0000 7fd687ae6640 1 --2- 172.21.15.98:0/3560573704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd68006e0a0 0x7fd680153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.249+0000 7fd687ae6640 1 -- 172.21.15.98:0/3560573704 >> 172.21.15.98:0/3560573704 conn(0x7fd6800d1bf0 msgr2=0x7fd680160ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.249+0000 7fd687ae6640 1 -- 172.21.15.98:0/3560573704 shutdown_connections 2024-09-10T20:58:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.250+0000 7fd687ae6640 1 -- 172.21.15.98:0/3560573704 wait complete. 2024-09-10T20:58:50.273 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-10T20:58:50.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.379+0000 7ff99e556640 1 -- 172.21.15.98:0/1484384513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99815be00 msgr2=0x7ff998070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:50.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.379+0000 7ff99e556640 1 --2- 172.21.15.98:0/1484384513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99815be00 0x7ff998070760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff988009830 tx=0x7ff98802a510 comp rx=0 tx=0).stop 2024-09-10T20:58:50.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.379+0000 7ff99e556640 1 -- 172.21.15.98:0/1484384513 shutdown_connections 2024-09-10T20:58:50.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.379+0000 7ff99e556640 1 --2- 172.21.15.98:0/1484384513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9980d0a00 0x7ff9980cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:50.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.379+0000 7ff99e556640 1 --2- 172.21.15.98:0/1484384513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99815be00 0x7ff998070760 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:50.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.379+0000 7ff99e556640 1 -- 172.21.15.98:0/1484384513 >> 172.21.15.98:0/1484384513 conn(0x7ff9980d1d10 msgr2=0x7ff9980d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:50.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.379+0000 7ff99e556640 1 -- 172.21.15.98:0/1484384513 shutdown_connections 2024-09-10T20:58:50.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.379+0000 7ff99e556640 1 -- 172.21.15.98:0/1484384513 wait complete. 2024-09-10T20:58:50.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.380+0000 7ff99e556640 1 Processor -- start 2024-09-10T20:58:50.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.380+0000 7ff99e556640 1 -- start start 2024-09-10T20:58:50.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.381+0000 7ff99e556640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9980d0a00 0x7ff9980b7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:50.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.381+0000 7ff99e556640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9980d2830 0x7ff99806f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:50.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.381+0000 7ff99e556640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99815be00 0x7ff998153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:50.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.381+0000 7ff99e556640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff9980cd1d0 con 0x7ff99815be00 2024-09-10T20:58:50.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.381+0000 7ff99e556640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff9980cd050 con 0x7ff9980d2830 2024-09-10T20:58:50.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.381+0000 7ff99e556640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff9980cd350 con 0x7ff9980d0a00 2024-09-10T20:58:50.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.381+0000 7ff99caf3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99815be00 0x7ff998153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:50.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.381+0000 7ff99caf3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99815be00 0x7ff998153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51856/0 (socket says 172.21.15.98:51856) 2024-09-10T20:58:50.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.381+0000 7ff99caf3640 1 -- 172.21.15.98:0/3019865790 learned_addr learned my addr 172.21.15.98:0/3019865790 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:50.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.381+0000 7ff997fff640 1 --2- 172.21.15.98:0/3019865790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9980d0a00 0x7ff9980b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:50.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.381+0000 7ff9977fe640 1 --2- 172.21.15.98:0/3019865790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9980d2830 0x7ff99806f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:50.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.382+0000 7ff99caf3640 1 -- 172.21.15.98:0/3019865790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9980d0a00 msgr2=0x7ff9980b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:50.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.382+0000 7ff99caf3640 1 --2- 172.21.15.98:0/3019865790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9980d0a00 0x7ff9980b7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:50.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.382+0000 7ff99caf3640 1 -- 172.21.15.98:0/3019865790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9980d2830 msgr2=0x7ff99806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:50.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.382+0000 7ff99caf3640 1 --2- 172.21.15.98:0/3019865790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9980d2830 0x7ff99806f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:50.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.382+0000 7ff99caf3640 1 -- 172.21.15.98:0/3019865790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff998154090 con 0x7ff99815be00 2024-09-10T20:58:50.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.382+0000 7ff99caf3640 1 --2- 172.21.15.98:0/3019865790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99815be00 0x7ff998153b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff99000b9e0 tx=0x7ff99000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:50.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.382+0000 7ff994ff9640 1 -- 172.21.15.98:0/3019865790 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990019070 con 0x7ff99815be00 2024-09-10T20:58:50.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.382+0000 7ff994ff9640 1 -- 172.21.15.98:0/3019865790 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff99000cc30 con 0x7ff99815be00 2024-09-10T20:58:50.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.382+0000 7ff99e556640 1 -- 172.21.15.98:0/3019865790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9981f5710 con 0x7ff99815be00 2024-09-10T20:58:50.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.382+0000 7ff994ff9640 1 -- 172.21.15.98:0/3019865790 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9900146a0 con 0x7ff99815be00 2024-09-10T20:58:50.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.382+0000 7ff99e556640 1 -- 172.21.15.98:0/3019865790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99806ddd0 con 0x7ff99815be00 2024-09-10T20:58:50.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.383+0000 7ff99e556640 1 -- 172.21.15.98:0/3019865790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9980cf8e0 con 0x7ff99815be00 2024-09-10T20:58:50.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.384+0000 7ff994ff9640 1 -- 172.21.15.98:0/3019865790 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff99001d3b0 con 0x7ff99815be00 2024-09-10T20:58:50.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.384+0000 7ff994ff9640 1 --2- 172.21.15.98:0/3019865790 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff984077f70 0x7ff98407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:50.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.384+0000 7ff994ff9640 1 -- 172.21.15.98:0/3019865790 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7ff99009a000 con 0x7ff99815be00 2024-09-10T20:58:50.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.384+0000 7ff997fff640 1 --2- 172.21.15.98:0/3019865790 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff984077f70 0x7ff98407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:50.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.385+0000 7ff997fff640 1 --2- 172.21.15.98:0/3019865790 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff984077f70 0x7ff98407a430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff988034000 tx=0x7ff988004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:50.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.389+0000 7ff994ff9640 1 -- 172.21.15.98:0/3019865790 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff99000ce20 con 0x7ff99815be00 2024-09-10T20:58:50.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:50.563+0000 7ff99e556640 1 -- 172.21.15.98:0/3019865790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7ff9980b7c90 con 0x7ff99815be00 2024-09-10T20:58:51.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.249+0000 7ff994ff9640 1 -- 172.21.15.98:0/3019865790 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7ff990062190 con 0x7ff99815be00 2024-09-10T20:58:51.254 INFO:teuthology.orchestra.run.smithi098.stderr:set pool 4 allow_ec_overwrites to true 2024-09-10T20:58:51.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.252+0000 7ff99e556640 1 -- 172.21.15.98:0/3019865790 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff984077f70 msgr2=0x7ff98407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:51.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.252+0000 7ff99e556640 1 --2- 172.21.15.98:0/3019865790 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff984077f70 0x7ff98407a430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff988034000 tx=0x7ff988004730 comp rx=0 tx=0).stop 2024-09-10T20:58:51.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.252+0000 7ff99e556640 1 -- 172.21.15.98:0/3019865790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99815be00 msgr2=0x7ff998153b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:51.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.252+0000 7ff99e556640 1 --2- 172.21.15.98:0/3019865790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99815be00 0x7ff998153b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff99000b9e0 tx=0x7ff99000bea0 comp rx=0 tx=0).stop 2024-09-10T20:58:51.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.252+0000 7ff99e556640 1 -- 172.21.15.98:0/3019865790 shutdown_connections 2024-09-10T20:58:51.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.252+0000 7ff99e556640 1 --2- 172.21.15.98:0/3019865790 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff984077f70 0x7ff98407a430 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:51.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.252+0000 7ff99e556640 1 --2- 172.21.15.98:0/3019865790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99815be00 0x7ff998153b50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:51.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.252+0000 7ff99e556640 1 --2- 172.21.15.98:0/3019865790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9980d2830 0x7ff99806f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:51.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.252+0000 7ff99e556640 1 --2- 172.21.15.98:0/3019865790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9980d0a00 0x7ff9980b7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:51.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.252+0000 7ff99e556640 1 -- 172.21.15.98:0/3019865790 >> 172.21.15.98:0/3019865790 conn(0x7ff9980d1d10 msgr2=0x7ff9980715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:51.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.252+0000 7ff99e556640 1 -- 172.21.15.98:0/3019865790 shutdown_connections 2024-09-10T20:58:51.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.252+0000 7ff99e556640 1 -- 172.21.15.98:0/3019865790 wait complete. 2024-09-10T20:58:51.275 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-10T20:58:51.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.377+0000 7f26809f4640 1 -- 172.21.15.98:0/3167140679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f267c0cf210 msgr2=0x7f267c072aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:51.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.377+0000 7f26809f4640 1 --2- 172.21.15.98:0/3167140679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f267c0cf210 0x7f267c072aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2674009830 tx=0x7f267402a480 comp rx=0 tx=0).stop 2024-09-10T20:58:51.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.377+0000 7f26809f4640 1 -- 172.21.15.98:0/3167140679 shutdown_connections 2024-09-10T20:58:51.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.377+0000 7f26809f4640 1 --2- 172.21.15.98:0/3167140679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f267c0cf210 0x7f267c072aa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:51.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.377+0000 7f26809f4640 1 --2- 172.21.15.98:0/3167140679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f267c144780 0x7f267c0b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:51.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.377+0000 7f26809f4640 1 -- 172.21.15.98:0/3167140679 >> 172.21.15.98:0/3167140679 conn(0x7f267c0b1760 msgr2=0x7f267c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:51.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.377+0000 7f26809f4640 1 -- 172.21.15.98:0/3167140679 shutdown_connections 2024-09-10T20:58:51.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.377+0000 7f26809f4640 1 -- 172.21.15.98:0/3167140679 wait complete. 2024-09-10T20:58:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.378+0000 7f26809f4640 1 Processor -- start 2024-09-10T20:58:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.378+0000 7f26809f4640 1 -- start start 2024-09-10T20:58:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f26809f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f267c072680 0x7f267c0af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f26809f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f267c0cf210 0x7f267c0b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f26809f4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f267c144780 0x7f267c141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f26809f4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f267c15c9c0 con 0x7f267c144780 2024-09-10T20:58:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f26809f4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f267c15c840 con 0x7f267c072680 2024-09-10T20:58:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f26809f4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f267c15cb40 con 0x7f267c0cf210 2024-09-10T20:58:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f267ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f267c144780 0x7f267c141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f267ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f267c144780 0x7f267c141fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51880/0 (socket says 172.21.15.98:51880) 2024-09-10T20:58:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f267ad9d640 1 -- 172.21.15.98:0/3466718922 learned_addr learned my addr 172.21.15.98:0/3466718922 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f2679d9b640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f267c0cf210 0x7f267c0b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f267a59c640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f267c072680 0x7f267c0af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f267ad9d640 1 -- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f267c0cf210 msgr2=0x7f267c0b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f267ad9d640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f267c0cf210 0x7f267c0b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f267ad9d640 1 -- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f267c072680 msgr2=0x7f267c0af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f267ad9d640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f267c072680 0x7f267c0af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f267ad9d640 1 -- 172.21.15.98:0/3466718922 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c1f5200 con 0x7f267c144780 2024-09-10T20:58:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.379+0000 7f267a59c640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f267c072680 0x7f267c0af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:58:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.380+0000 7f267ad9d640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f267c144780 0x7f267c141fc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2674009920 tx=0x7f2674032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.380+0000 7f266affd640 1 -- 172.21.15.98:0/3466718922 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674040070 con 0x7f267c144780 2024-09-10T20:58:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.380+0000 7f266affd640 1 -- 172.21.15.98:0/3466718922 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2674005310 con 0x7f267c144780 2024-09-10T20:58:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.380+0000 7f26809f4640 1 -- 172.21.15.98:0/3466718922 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c1f54e0 con 0x7f267c144780 2024-09-10T20:58:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.380+0000 7f266affd640 1 -- 172.21.15.98:0/3466718922 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26740047a0 con 0x7f267c144780 2024-09-10T20:58:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.380+0000 7f26809f4640 1 -- 172.21.15.98:0/3466718922 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c1f5aa0 con 0x7f267c144780 2024-09-10T20:58:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.381+0000 7f26809f4640 1 -- 172.21.15.98:0/3466718922 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267c0b3da0 con 0x7f267c144780 2024-09-10T20:58:51.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.381+0000 7f266affd640 1 -- 172.21.15.98:0/3466718922 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f267404c030 con 0x7f267c144780 2024-09-10T20:58:51.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.382+0000 7f266affd640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f265c077f70 0x7f265c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:51.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.382+0000 7f266affd640 1 -- 172.21.15.98:0/3466718922 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f26740bb010 con 0x7f267c144780 2024-09-10T20:58:51.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.382+0000 7f267a59c640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f265c077f70 0x7f265c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:51.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.383+0000 7f267a59c640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f265c077f70 0x7f265c07a430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f266c005fd0 tx=0x7f266c009480 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:51.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.386+0000 7f266affd640 1 -- 172.21.15.98:0/3466718922 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2674083880 con 0x7f267c144780 2024-09-10T20:58:51.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:51.601+0000 7f26809f4640 1 -- 172.21.15.98:0/3466718922 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f267c0b1f30 con 0x7f267c144780 2024-09-10T20:58:52.201 INFO:tasks.ceph.mds.c.smithi098.stderr:2024-09-10T20:58:52.197+0000 7fc5824a7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T20:58:52.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.251+0000 7f266affd640 1 -- 172.21.15.98:0/3466718922 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) ==== 137+0+0 (secure 0 0 0) 0x7f2674083180 con 0x7f267c144780 2024-09-10T20:58:52.255 INFO:teuthology.orchestra.run.smithi098.stderr:added data pool 4 to fsmap 2024-09-10T20:58:52.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.253+0000 7f26809f4640 1 -- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f265c077f70 msgr2=0x7f265c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:52.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.253+0000 7f26809f4640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f265c077f70 0x7f265c07a430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f266c005fd0 tx=0x7f266c009480 comp rx=0 tx=0).stop 2024-09-10T20:58:52.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.253+0000 7f26809f4640 1 -- 172.21.15.98:0/3466718922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f267c144780 msgr2=0x7f267c141fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:52.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.253+0000 7f26809f4640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f267c144780 0x7f267c141fc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2674009920 tx=0x7f2674032c10 comp rx=0 tx=0).stop 2024-09-10T20:58:52.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.253+0000 7f26809f4640 1 -- 172.21.15.98:0/3466718922 shutdown_connections 2024-09-10T20:58:52.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.253+0000 7f26809f4640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f265c077f70 0x7f265c07a430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.253+0000 7f26809f4640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f267c144780 0x7f267c141fc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.253+0000 7f26809f4640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f267c0cf210 0x7f267c0b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.253+0000 7f26809f4640 1 --2- 172.21.15.98:0/3466718922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f267c072680 0x7f267c0af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.253+0000 7f26809f4640 1 -- 172.21.15.98:0/3466718922 >> 172.21.15.98:0/3466718922 conn(0x7f267c0b1760 msgr2=0x7f267c06ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:52.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.253+0000 7f26809f4640 1 -- 172.21.15.98:0/3466718922 shutdown_connections 2024-09-10T20:58:52.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.253+0000 7f26809f4640 1 -- 172.21.15.98:0/3466718922 wait complete. 2024-09-10T20:58:52.271 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:58:52.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.370+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/3916191439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7815bb60 msgr2=0x7f2f780b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:52.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.370+0000 7f2f7d6d4640 1 --2- 172.21.15.98:0/3916191439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7815bb60 0x7f2f780b3a70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2f780b3480 tx=0x7f2f7002a560 comp rx=0 tx=0).stop 2024-09-10T20:58:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.370+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/3916191439 shutdown_connections 2024-09-10T20:58:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.370+0000 7f2f7d6d4640 1 --2- 172.21.15.98:0/3916191439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7815bb60 0x7f2f780b3a70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.370+0000 7f2f7d6d4640 1 --2- 172.21.15.98:0/3916191439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f78073a30 0x7f2f7815b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.370+0000 7f2f7d6d4640 1 --2- 172.21.15.98:0/3916191439 >> v2:172.21.15.190:3300/0 conn(0x7f2f78073650 0x7f2f7815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.370+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/3916191439 >> 172.21.15.98:0/3916191439 conn(0x7f2f780b2770 msgr2=0x7f2f780b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.370+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/3916191439 shutdown_connections 2024-09-10T20:58:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.370+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/3916191439 wait complete. 2024-09-10T20:58:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.371+0000 7f2f7d6d4640 1 Processor -- start 2024-09-10T20:58:52.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.371+0000 7f2f7d6d4640 1 -- start start 2024-09-10T20:58:52.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.372+0000 7f2f7d6d4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f78073650 0x7f2f7813ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:52.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.372+0000 7f2f7d6d4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f78073a30 0x7f2f78142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:52.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.372+0000 7f2f7d6d4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7815a1c0 0x7f2f7813cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:52.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.372+0000 7f2f7d6d4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2f780709f0 con 0x7f2f7815a1c0 2024-09-10T20:58:52.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.372+0000 7f2f7d6d4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2f78070870 con 0x7f2f78073650 2024-09-10T20:58:52.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.372+0000 7f2f7d6d4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2f780b3fb0 con 0x7f2f78073a30 2024-09-10T20:58:52.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.372+0000 7f2f777fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7815a1c0 0x7f2f7813cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:52.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.372+0000 7f2f777fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7815a1c0 0x7f2f7813cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51884/0 (socket says 172.21.15.98:51884) 2024-09-10T20:58:52.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.372+0000 7f2f777fe640 1 -- 172.21.15.98:0/2095432826 learned_addr learned my addr 172.21.15.98:0/2095432826 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:52.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.372+0000 7f2f76ffd640 1 --2- 172.21.15.98:0/2095432826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f78073650 0x7f2f7813ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:52.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.372+0000 7f2f767fc640 1 --2- 172.21.15.98:0/2095432826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f78073a30 0x7f2f78142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:52.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.373+0000 7f2f777fe640 1 -- 172.21.15.98:0/2095432826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f78073a30 msgr2=0x7f2f78142fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:52.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.373+0000 7f2f777fe640 1 --2- 172.21.15.98:0/2095432826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f78073a30 0x7f2f78142fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.373+0000 7f2f777fe640 1 -- 172.21.15.98:0/2095432826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f78073650 msgr2=0x7f2f7813ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:52.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.373+0000 7f2f777fe640 1 --2- 172.21.15.98:0/2095432826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f78073650 0x7f2f7813ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.373+0000 7f2f777fe640 1 -- 172.21.15.98:0/2095432826 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f781f5900 con 0x7f2f7815a1c0 2024-09-10T20:58:52.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.373+0000 7f2f777fe640 1 --2- 172.21.15.98:0/2095432826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7815a1c0 0x7f2f7813cf10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009230 tx=0x7f2f70003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:52.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.373+0000 7f2f6f7fe640 1 -- 172.21.15.98:0/2095432826 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7003e030 con 0x7f2f7815a1c0 2024-09-10T20:58:52.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.373+0000 7f2f6f7fe640 1 -- 172.21.15.98:0/2095432826 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f70003e50 con 0x7f2f7815a1c0 2024-09-10T20:58:52.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.373+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/2095432826 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f7813f580 con 0x7f2f7815a1c0 2024-09-10T20:58:52.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.373+0000 7f2f6f7fe640 1 -- 172.21.15.98:0/2095432826 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70035040 con 0x7f2f7815a1c0 2024-09-10T20:58:52.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.373+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/2095432826 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f7813fb40 con 0x7f2f7815a1c0 2024-09-10T20:58:52.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.374+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/2095432826 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f7815b620 con 0x7f2f7815a1c0 2024-09-10T20:58:52.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.374+0000 7f2f6f7fe640 1 -- 172.21.15.98:0/2095432826 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2f70004a00 con 0x7f2f7815a1c0 2024-09-10T20:58:52.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.375+0000 7f2f6f7fe640 1 --2- 172.21.15.98:0/2095432826 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2f58077fa0 0x7f2f5807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:52.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.375+0000 7f2f6f7fe640 1 -- 172.21.15.98:0/2095432826 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2f700ba950 con 0x7f2f7815a1c0 2024-09-10T20:58:52.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.375+0000 7f2f76ffd640 1 --2- 172.21.15.98:0/2095432826 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2f58077fa0 0x7f2f5807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:52.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.376+0000 7f2f76ffd640 1 --2- 172.21.15.98:0/2095432826 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2f58077fa0 0x7f2f5807a460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2f68005c40 tx=0x7f2f6800a590 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:52.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.379+0000 7f2f6f7fe640 1 -- 172.21.15.98:0/2095432826 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2f70082ac0 con 0x7f2f7815a1c0 2024-09-10T20:58:52.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.589+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/2095432826 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f780d0850 con 0x7f2f7815a1c0 2024-09-10T20:58:52.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.589+0000 7f2f6f7fe640 1 -- 172.21.15.98:0/2095432826 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5216 (secure 0 0 0) 0x7f2f700874f0 con 0x7f2f7815a1c0 2024-09-10T20:58:52.593 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:58:52.594 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":6,"btime":"2024-09-10T20:58:52:250780+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4277,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/1954552281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1954552281},{"type":"v1","addr":"172.21.15.190:6836","nonce":1954552281}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/8997965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":8997965},{"type":"v1","addr":"172.21.15.190:6837","nonce":8997965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1984550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1984550306},{"type":"v1","addr":"172.21.15.98:6833","nonce":1984550306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:58:46.560479+0000","modified":"2024-09-10T20:58:51.602516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3141703317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3141703317},{"type":"v1","addr":"172.21.15.98:6835","nonce":3141703317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T20:58:52.594 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 6 2024-09-10T20:58:52.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.591+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/2095432826 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2f58077fa0 msgr2=0x7f2f5807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:52.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.591+0000 7f2f7d6d4640 1 --2- 172.21.15.98:0/2095432826 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2f58077fa0 0x7f2f5807a460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2f68005c40 tx=0x7f2f6800a590 comp rx=0 tx=0).stop 2024-09-10T20:58:52.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.591+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/2095432826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7815a1c0 msgr2=0x7f2f7813cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:52.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.591+0000 7f2f7d6d4640 1 --2- 172.21.15.98:0/2095432826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7815a1c0 0x7f2f7813cf10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009230 tx=0x7f2f70003960 comp rx=0 tx=0).stop 2024-09-10T20:58:52.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.591+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/2095432826 shutdown_connections 2024-09-10T20:58:52.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.591+0000 7f2f7d6d4640 1 --2- 172.21.15.98:0/2095432826 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2f58077fa0 0x7f2f5807a460 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.591+0000 7f2f7d6d4640 1 --2- 172.21.15.98:0/2095432826 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f7815a1c0 0x7f2f7813cf10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.591+0000 7f2f7d6d4640 1 --2- 172.21.15.98:0/2095432826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f78073a30 0x7f2f78142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.591+0000 7f2f7d6d4640 1 --2- 172.21.15.98:0/2095432826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f78073650 0x7f2f7813ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.591+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/2095432826 >> 172.21.15.98:0/2095432826 conn(0x7f2f780b2770 msgr2=0x7f2f780b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:52.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.591+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/2095432826 shutdown_connections 2024-09-10T20:58:52.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.591+0000 7f2f7d6d4640 1 -- 172.21.15.98:0/2095432826 wait complete. 2024-09-10T20:58:52.606 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:58:52.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.702+0000 7f2c81bda640 1 -- 172.21.15.98:0/4000378124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c64003a50 msgr2=0x7f2c64003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:52.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.702+0000 7f2c81bda640 1 --2- 172.21.15.98:0/4000378124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c64003a50 0x7f2c64003e70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009ed0 tx=0x7f2c64005830 comp rx=0 tx=0).stop 2024-09-10T20:58:52.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.702+0000 7f2c81bda640 1 -- 172.21.15.98:0/4000378124 shutdown_connections 2024-09-10T20:58:52.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.702+0000 7f2c81bda640 1 --2- 172.21.15.98:0/4000378124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c64003a50 0x7f2c64003e70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.702+0000 7f2c81bda640 1 -- 172.21.15.98:0/4000378124 >> 172.21.15.98:0/4000378124 conn(0x7f2c7c0d0c00 msgr2=0x7f2c7c0d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:52.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.702+0000 7f2c81bda640 1 -- 172.21.15.98:0/4000378124 shutdown_connections 2024-09-10T20:58:52.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.702+0000 7f2c81bda640 1 -- 172.21.15.98:0/4000378124 wait complete. 2024-09-10T20:58:52.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.704+0000 7f2c81bda640 1 Processor -- start 2024-09-10T20:58:52.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.704+0000 7f2c81bda640 1 -- start start 2024-09-10T20:58:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c81bda640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c64003a50 0x7f2c7c0a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c81bda640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c7c14fa30 0x7f2c7c14b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c81bda640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c7c1f9260 0x7f2c7c1e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c81bda640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c0b4180 con 0x7f2c7c14fa30 2024-09-10T20:58:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c81bda640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c0b4000 con 0x7f2c7c1f9260 2024-09-10T20:58:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c81bda640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2c7c0b4300 con 0x7f2c64003a50 2024-09-10T20:58:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c7affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c7c14fa30 0x7f2c7c14b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c7affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c7c14fa30 0x7f2c7c14b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51902/0 (socket says 172.21.15.98:51902) 2024-09-10T20:58:52.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c7affd640 1 -- 172.21.15.98:0/1280946452 learned_addr learned my addr 172.21.15.98:0/1280946452 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c7bfff640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c7c1f9260 0x7f2c7c1e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c7b7fe640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c64003a50 0x7f2c7c0a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c7affd640 1 -- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c64003a50 msgr2=0x7f2c7c0a96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c7affd640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c64003a50 0x7f2c7c0a96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c7affd640 1 -- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c7c1f9260 msgr2=0x7f2c7c1e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c7affd640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c7c1f9260 0x7f2c7c1e8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c7affd640 1 -- 172.21.15.98:0/1280946452 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7c0a9ee0 con 0x7f2c7c14fa30 2024-09-10T20:58:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c7b7fe640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c64003a50 0x7f2c7c0a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:58:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c7affd640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c7c14fa30 0x7f2c7c14b280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c70004db0 tx=0x7f2c70001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c63fff640 1 -- 172.21.15.98:0/1280946452 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70001d40 con 0x7f2c7c14fa30 2024-09-10T20:58:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c81bda640 1 -- 172.21.15.98:0/1280946452 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7c1f8780 con 0x7f2c7c14fa30 2024-09-10T20:58:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c63fff640 1 -- 172.21.15.98:0/1280946452 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c70016070 con 0x7f2c7c14fa30 2024-09-10T20:58:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c63fff640 1 -- 172.21.15.98:0/1280946452 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70010b40 con 0x7f2c7c14fa30 2024-09-10T20:58:52.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.705+0000 7f2c81bda640 1 -- 172.21.15.98:0/1280946452 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7c1f8d40 con 0x7f2c7c14fa30 2024-09-10T20:58:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.706+0000 7f2c81bda640 1 -- 172.21.15.98:0/1280946452 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c7c071a90 con 0x7f2c7c14fa30 2024-09-10T20:58:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.707+0000 7f2c63fff640 1 -- 172.21.15.98:0/1280946452 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2c70010ce0 con 0x7f2c7c14fa30 2024-09-10T20:58:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.707+0000 7f2c63fff640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2c68077fd0 0x7f2c6807a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.707+0000 7f2c63fff640 1 -- 172.21.15.98:0/1280946452 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2c7009a7c0 con 0x7f2c7c14fa30 2024-09-10T20:58:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.707+0000 7f2c7b7fe640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2c68077fd0 0x7f2c6807a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:52.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.708+0000 7f2c7b7fe640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2c68077fd0 0x7f2c6807a490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c035000 tx=0x7f2c6c004a40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:52.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.711+0000 7f2c63fff640 1 -- 172.21.15.98:0/1280946452 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2c7c071a90 con 0x7f2c7c14fa30 2024-09-10T20:58:52.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.887+0000 7f2c81bda640 1 -- 172.21.15.98:0/1280946452 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c7c15ed30 con 0x7f2c7c14fa30 2024-09-10T20:58:52.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.888+0000 7f2c63fff640 1 -- 172.21.15.98:0/1280946452 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18610 (secure 0 0 0) 0x7f2c70062f90 con 0x7f2c7c14fa30 2024-09-10T20:58:52.893 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:58:52.894 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":25,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:58:51.602367+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:58:44.390290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T20:58:45.564099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T20:58:49.250508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:58:52.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.890+0000 7f2c81bda640 1 -- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2c68077fd0 msgr2=0x7f2c6807a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:52.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.890+0000 7f2c81bda640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2c68077fd0 0x7f2c6807a490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c035000 tx=0x7f2c6c004a40 comp rx=0 tx=0).stop 2024-09-10T20:58:52.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.890+0000 7f2c81bda640 1 -- 172.21.15.98:0/1280946452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c7c14fa30 msgr2=0x7f2c7c14b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:52.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.890+0000 7f2c81bda640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c7c14fa30 0x7f2c7c14b280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c70004db0 tx=0x7f2c70001630 comp rx=0 tx=0).stop 2024-09-10T20:58:52.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.890+0000 7f2c81bda640 1 -- 172.21.15.98:0/1280946452 shutdown_connections 2024-09-10T20:58:52.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.890+0000 7f2c81bda640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2c68077fd0 0x7f2c6807a490 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.890+0000 7f2c81bda640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c7c1f9260 0x7f2c7c1e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.890+0000 7f2c81bda640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c7c14fa30 0x7f2c7c14b280 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.890+0000 7f2c81bda640 1 --2- 172.21.15.98:0/1280946452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c64003a50 0x7f2c7c0a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:52.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.890+0000 7f2c81bda640 1 -- 172.21.15.98:0/1280946452 >> 172.21.15.98:0/1280946452 conn(0x7f2c7c0d0c00 msgr2=0x7f2c7c0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:52.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.891+0000 7f2c81bda640 1 -- 172.21.15.98:0/1280946452 shutdown_connections 2024-09-10T20:58:52.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:52.891+0000 7f2c81bda640 1 -- 172.21.15.98:0/1280946452 wait complete. 2024-09-10T20:58:52.905 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:58:53.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.002+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/3874953943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f98003fd0 msgr2=0x7f7f98004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.002+0000 7f7fb5ff3640 1 --2- 172.21.15.98:0/3874953943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7f98003fd0 0x7f7f98004800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400c290 tx=0x7f7f98005830 comp rx=0 tx=0).stop 2024-09-10T20:58:53.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.002+0000 7f7faffff640 1 -- 172.21.15.98:0/3874953943 reap_dead start 2024-09-10T20:58:53.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.002+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/3874953943 shutdown_connections 2024-09-10T20:58:53.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.002+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/3874953943 >> 172.21.15.98:0/3874953943 conn(0x7f7fb00b1c80 msgr2=0x7f7fb00b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:53.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.002+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/3874953943 shutdown_connections 2024-09-10T20:58:53.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.002+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/3874953943 wait complete. 2024-09-10T20:58:53.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.003+0000 7f7fb5ff3640 1 Processor -- start 2024-09-10T20:58:53.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.004+0000 7f7fb5ff3640 1 -- start start 2024-09-10T20:58:53.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.004+0000 7f7fb5ff3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f98003fd0 0x7f7fb01f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:53.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.004+0000 7f7fb5ff3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fb0156180 0x7f7fb00ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:53.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.004+0000 7f7fb5ff3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb0152cd0 0x7f7fb0153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:53.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.004+0000 7f7fb5ff3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb015b4a0 con 0x7f7fb0152cd0 2024-09-10T20:58:53.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.004+0000 7f7fb5ff3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb015b320 con 0x7f7fb0156180 2024-09-10T20:58:53.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.004+0000 7f7fb5ff3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7fb015b620 con 0x7f7f98003fd0 2024-09-10T20:58:53.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.004+0000 7f7faffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb0152cd0 0x7f7fb0153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:53.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.004+0000 7f7faffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb0152cd0 0x7f7fb0153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51910/0 (socket says 172.21.15.98:51910) 2024-09-10T20:58:53.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.004+0000 7f7faffff640 1 -- 172.21.15.98:0/648952732 learned_addr learned my addr 172.21.15.98:0/648952732 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:53.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.005+0000 7f7faeffd640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fb0156180 0x7f7fb00ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:53.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.005+0000 7f7faf7fe640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f98003fd0 0x7f7fb01f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:53.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.005+0000 7f7faffff640 1 -- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f98003fd0 msgr2=0x7f7fb01f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.005+0000 7f7faffff640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f98003fd0 0x7f7fb01f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.005+0000 7f7faffff640 1 -- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fb0156180 msgr2=0x7f7fb00ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.005+0000 7f7faffff640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fb0156180 0x7f7fb00ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.005+0000 7f7faffff640 1 -- 172.21.15.98:0/648952732 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb01f1610 con 0x7f7fb0152cd0 2024-09-10T20:58:53.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.005+0000 7f7faeffd640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fb0156180 0x7f7fb00ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:58:53.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.005+0000 7f7faffff640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb0152cd0 0x7f7fb0153150 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0072510 tx=0x7f7fa8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:53.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.005+0000 7f7f97fff640 1 -- 172.21.15.98:0/648952732 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa80035f0 con 0x7f7fb0152cd0 2024-09-10T20:58:53.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.005+0000 7f7f97fff640 1 -- 172.21.15.98:0/648952732 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fa8016070 con 0x7f7fb0152cd0 2024-09-10T20:58:53.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.005+0000 7f7f97fff640 1 -- 172.21.15.98:0/648952732 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8003b00 con 0x7f7fb0152cd0 2024-09-10T20:58:53.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.005+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/648952732 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb01f18d0 con 0x7f7fb0152cd0 2024-09-10T20:58:53.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.005+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/648952732 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb01f1e90 con 0x7f7fb0152cd0 2024-09-10T20:58:53.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.006+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/648952732 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fb0150340 con 0x7f7fb0152cd0 2024-09-10T20:58:53.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.007+0000 7f7f97fff640 1 -- 172.21.15.98:0/648952732 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7fa8023030 con 0x7f7fb0152cd0 2024-09-10T20:58:53.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.007+0000 7f7f97fff640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7f9c077f70 0x7f7f9c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:53.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.007+0000 7f7f97fff640 1 -- 172.21.15.98:0/648952732 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7fa809a550 con 0x7f7fb0152cd0 2024-09-10T20:58:53.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.007+0000 7f7faf7fe640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7f9c077f70 0x7f7f9c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:53.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.008+0000 7f7faf7fe640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7f9c077f70 0x7f7f9c07a430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0005fd0 tx=0x7f7fa0009500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:53.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.011+0000 7f7f97fff640 1 -- 172.21.15.98:0/648952732 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7fa8062da0 con 0x7f7fb0152cd0 2024-09-10T20:58:53.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.185+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/648952732 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7fb00b4340 con 0x7f7fb0152cd0 2024-09-10T20:58:53.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.186+0000 7f7f97fff640 1 -- 172.21.15.98:0/648952732 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18610 (secure 0 0 0) 0x7f7fa80626a0 con 0x7f7fb0152cd0 2024-09-10T20:58:53.190 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:58:53.191 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":25,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:58:51.602367+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:58:44.390290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T20:58:45.564099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T20:58:49.250508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:58:53.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.188+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7f9c077f70 msgr2=0x7f7f9c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.188+0000 7f7fb5ff3640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7f9c077f70 0x7f7f9c07a430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0005fd0 tx=0x7f7fa0009500 comp rx=0 tx=0).stop 2024-09-10T20:58:53.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.188+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/648952732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb0152cd0 msgr2=0x7f7fb0153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.188+0000 7f7fb5ff3640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb0152cd0 0x7f7fb0153150 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0072510 tx=0x7f7fa8011aa0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.188+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/648952732 shutdown_connections 2024-09-10T20:58:53.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.188+0000 7f7fb5ff3640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7f9c077f70 0x7f7f9c07a430 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.188+0000 7f7fb5ff3640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fb0152cd0 0x7f7fb0153150 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.188+0000 7f7fb5ff3640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fb0156180 0x7f7fb00ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.188+0000 7f7fb5ff3640 1 --2- 172.21.15.98:0/648952732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f98003fd0 0x7f7fb01f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.188+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/648952732 >> 172.21.15.98:0/648952732 conn(0x7f7fb00b1c80 msgr2=0x7f7fb00b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:53.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.188+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/648952732 shutdown_connections 2024-09-10T20:58:53.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.189+0000 7f7fb5ff3640 1 -- 172.21.15.98:0/648952732 wait complete. 2024-09-10T20:58:53.221 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:58:53.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.318+0000 7fad49ad4640 1 -- 172.21.15.98:0/2584848697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad440d2a70 msgr2=0x7fad440b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.318+0000 7fad49ad4640 1 --2- 172.21.15.98:0/2584848697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad440d2a70 0x7fad440b3a30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad440d3dc0 tx=0x7fad3802a4a0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.318+0000 7fad49ad4640 1 -- 172.21.15.98:0/2584848697 shutdown_connections 2024-09-10T20:58:53.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.318+0000 7fad49ad4640 1 --2- 172.21.15.98:0/2584848697 >> v2:172.21.15.190:3300/0 conn(0x7fad440b55a0 0x7fad440b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.318+0000 7fad49ad4640 1 --2- 172.21.15.98:0/2584848697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad440d2a70 0x7fad440b3a30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.318+0000 7fad49ad4640 1 -- 172.21.15.98:0/2584848697 >> 172.21.15.98:0/2584848697 conn(0x7fad4409d170 msgr2=0x7fad440d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:53.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.318+0000 7fad49ad4640 1 -- 172.21.15.98:0/2584848697 shutdown_connections 2024-09-10T20:58:53.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.318+0000 7fad49ad4640 1 -- 172.21.15.98:0/2584848697 wait complete. 2024-09-10T20:58:53.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.320+0000 7fad49ad4640 1 Processor -- start 2024-09-10T20:58:53.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.320+0000 7fad49ad4640 1 -- start start 2024-09-10T20:58:53.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.320+0000 7fad49ad4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad440b55a0 0x7fad440b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:53.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.320+0000 7fad49ad4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad440d2a70 0x7fad4409e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:53.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.320+0000 7fad49ad4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad44161560 0x7fad4415d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:53.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.320+0000 7fad49ad4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fad440d0450 con 0x7fad44161560 2024-09-10T20:58:53.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.320+0000 7fad49ad4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fad440d4830 con 0x7fad440d2a70 2024-09-10T20:58:53.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.320+0000 7fad49ad4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fad440b44b0 con 0x7fad440b55a0 2024-09-10T20:58:53.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad43fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad44161560 0x7fad4415d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:53.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad43fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad44161560 0x7fad4415d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51928/0 (socket says 172.21.15.98:51928) 2024-09-10T20:58:53.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad43fff640 1 -- 172.21.15.98:0/4270195832 learned_addr learned my addr 172.21.15.98:0/4270195832 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:53.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad42ffd640 1 --2- 172.21.15.98:0/4270195832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad440d2a70 0x7fad4409e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:53.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad437fe640 1 --2- 172.21.15.98:0/4270195832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad440b55a0 0x7fad440b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:53.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad43fff640 1 -- 172.21.15.98:0/4270195832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad440b55a0 msgr2=0x7fad440b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad43fff640 1 --2- 172.21.15.98:0/4270195832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad440b55a0 0x7fad440b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad43fff640 1 -- 172.21.15.98:0/4270195832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad440d2a70 msgr2=0x7fad4409e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad43fff640 1 --2- 172.21.15.98:0/4270195832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad440d2a70 0x7fad4409e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad43fff640 1 -- 172.21.15.98:0/4270195832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad440b2300 con 0x7fad44161560 2024-09-10T20:58:53.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad43fff640 1 --2- 172.21.15.98:0/4270195832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad44161560 0x7fad4415d3b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00ea00 tx=0x7fad3c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:53.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad2bfff640 1 -- 172.21.15.98:0/4270195832 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c004d20 con 0x7fad44161560 2024-09-10T20:58:53.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad2bfff640 1 -- 172.21.15.98:0/4270195832 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad3c00cb20 con 0x7fad44161560 2024-09-10T20:58:53.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad2bfff640 1 -- 172.21.15.98:0/4270195832 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c005020 con 0x7fad44161560 2024-09-10T20:58:53.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad49ad4640 1 -- 172.21.15.98:0/4270195832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad440ceea0 con 0x7fad44161560 2024-09-10T20:58:53.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.321+0000 7fad49ad4640 1 -- 172.21.15.98:0/4270195832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad440b25a0 con 0x7fad44161560 2024-09-10T20:58:53.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.322+0000 7fad49ad4640 1 -- 172.21.15.98:0/4270195832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4409dd00 con 0x7fad44161560 2024-09-10T20:58:53.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.323+0000 7fad2bfff640 1 -- 172.21.15.98:0/4270195832 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fad3c00f040 con 0x7fad44161560 2024-09-10T20:58:53.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.323+0000 7fad2bfff640 1 --2- 172.21.15.98:0/4270195832 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fad30077f70 0x7fad3007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:53.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.324+0000 7fad2bfff640 1 -- 172.21.15.98:0/4270195832 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fad3c09a620 con 0x7fad44161560 2024-09-10T20:58:53.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.324+0000 7fad437fe640 1 --2- 172.21.15.98:0/4270195832 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fad30077f70 0x7fad3007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:53.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.324+0000 7fad437fe640 1 --2- 172.21.15.98:0/4270195832 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fad30077f70 0x7fad3007a430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad34000f60 tx=0x7fad34009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:53.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.328+0000 7fad2bfff640 1 -- 172.21.15.98:0/4270195832 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fad3c016030 con 0x7fad44161560 2024-09-10T20:58:53.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.537+0000 7fad49ad4640 1 -- 172.21.15.98:0/4270195832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad440b4910 con 0x7fad44161560 2024-09-10T20:58:53.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.538+0000 7fad2bfff640 1 -- 172.21.15.98:0/4270195832 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5216 (secure 0 0 0) 0x7fad3c062790 con 0x7fad44161560 2024-09-10T20:58:53.542 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:58:53.542 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":6,"btime":"2024-09-10T20:58:52:250780+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4277,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/1954552281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1954552281},{"type":"v1","addr":"172.21.15.190:6836","nonce":1954552281}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/8997965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":8997965},{"type":"v1","addr":"172.21.15.190:6837","nonce":8997965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1984550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1984550306},{"type":"v1","addr":"172.21.15.98:6833","nonce":1984550306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:58:46.560479+0000","modified":"2024-09-10T20:58:51.602516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3141703317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3141703317},{"type":"v1","addr":"172.21.15.98:6835","nonce":3141703317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T20:58:53.542 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 6 2024-09-10T20:58:53.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.540+0000 7fad49ad4640 1 -- 172.21.15.98:0/4270195832 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fad30077f70 msgr2=0x7fad3007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.540+0000 7fad49ad4640 1 --2- 172.21.15.98:0/4270195832 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fad30077f70 0x7fad3007a430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad34000f60 tx=0x7fad34009040 comp rx=0 tx=0).stop 2024-09-10T20:58:53.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.540+0000 7fad49ad4640 1 -- 172.21.15.98:0/4270195832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad44161560 msgr2=0x7fad4415d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.540+0000 7fad49ad4640 1 --2- 172.21.15.98:0/4270195832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad44161560 0x7fad4415d3b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00ea00 tx=0x7fad3c00eec0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.540+0000 7fad49ad4640 1 -- 172.21.15.98:0/4270195832 shutdown_connections 2024-09-10T20:58:53.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.540+0000 7fad49ad4640 1 --2- 172.21.15.98:0/4270195832 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fad30077f70 0x7fad3007a430 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.540+0000 7fad49ad4640 1 --2- 172.21.15.98:0/4270195832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad44161560 0x7fad4415d3b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.540+0000 7fad49ad4640 1 --2- 172.21.15.98:0/4270195832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad440d2a70 0x7fad4409e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.540+0000 7fad49ad4640 1 --2- 172.21.15.98:0/4270195832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad440b55a0 0x7fad440b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.540+0000 7fad49ad4640 1 -- 172.21.15.98:0/4270195832 >> 172.21.15.98:0/4270195832 conn(0x7fad4409d170 msgr2=0x7fad4415ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:53.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.541+0000 7fad49ad4640 1 -- 172.21.15.98:0/4270195832 shutdown_connections 2024-09-10T20:58:53.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.541+0000 7fad49ad4640 1 -- 172.21.15.98:0/4270195832 wait complete. 2024-09-10T20:58:53.555 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:58:53.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.651+0000 7f0b182b8640 1 -- 172.21.15.98:0/842976179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b1006f140 msgr2=0x7f0b1006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.651+0000 7f0b182b8640 1 --2- 172.21.15.98:0/842976179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b1006f140 0x7f0b1006f5d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0b08005fd0 tx=0x7f0b0802aa90 comp rx=0 tx=0).stop 2024-09-10T20:58:53.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.651+0000 7f0b182b8640 1 -- 172.21.15.98:0/842976179 shutdown_connections 2024-09-10T20:58:53.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.651+0000 7f0b182b8640 1 --2- 172.21.15.98:0/842976179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b1006f140 0x7f0b1006f5d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.651+0000 7f0b182b8640 1 --2- 172.21.15.98:0/842976179 >> v2:172.21.15.190:3300/0 conn(0x7f0b10073780 0x7f0b10073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.651+0000 7f0b182b8640 1 -- 172.21.15.98:0/842976179 >> 172.21.15.98:0/842976179 conn(0x7f0b100b1760 msgr2=0x7f0b100b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:53.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.651+0000 7f0b182b8640 1 -- 172.21.15.98:0/842976179 shutdown_connections 2024-09-10T20:58:53.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.651+0000 7f0b182b8640 1 -- 172.21.15.98:0/842976179 wait complete. 2024-09-10T20:58:53.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.653+0000 7f0b182b8640 1 Processor -- start 2024-09-10T20:58:53.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.653+0000 7f0b182b8640 1 -- start start 2024-09-10T20:58:53.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.653+0000 7f0b182b8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b1006f140 0x7f0b100ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:53.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.653+0000 7f0b182b8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b10073780 0x7f0b100b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:53.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.653+0000 7f0b182b8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b1015c470 0x7f0b101f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:53.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.653+0000 7f0b182b8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10142260 con 0x7f0b1015c470 2024-09-10T20:58:53.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.653+0000 7f0b182b8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101420e0 con 0x7f0b10073780 2024-09-10T20:58:53.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.653+0000 7f0b182b8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1006fb10 con 0x7f0b1006f140 2024-09-10T20:58:53.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.653+0000 7f0b16855640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b1015c470 0x7f0b101f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:53.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.653+0000 7f0b16855640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b1015c470 0x7f0b101f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51934/0 (socket says 172.21.15.98:51934) 2024-09-10T20:58:53.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.654+0000 7f0b16855640 1 -- 172.21.15.98:0/1150160251 learned_addr learned my addr 172.21.15.98:0/1150160251 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:53.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.654+0000 7f0b16054640 1 --2- 172.21.15.98:0/1150160251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b1006f140 0x7f0b100ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:53.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.654+0000 7f0b15853640 1 --2- 172.21.15.98:0/1150160251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b10073780 0x7f0b100b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:53.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.654+0000 7f0b16855640 1 -- 172.21.15.98:0/1150160251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b1006f140 msgr2=0x7f0b100ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.654+0000 7f0b16855640 1 --2- 172.21.15.98:0/1150160251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b1006f140 0x7f0b100ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.654+0000 7f0b16855640 1 -- 172.21.15.98:0/1150160251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b10073780 msgr2=0x7f0b100b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.654+0000 7f0b16855640 1 --2- 172.21.15.98:0/1150160251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b10073780 0x7f0b100b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.654+0000 7f0b16855640 1 -- 172.21.15.98:0/1150160251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b101f5840 con 0x7f0b1015c470 2024-09-10T20:58:53.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.654+0000 7f0b16855640 1 --2- 172.21.15.98:0/1150160251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b1015c470 0x7f0b101f48c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009bf0 tx=0x7f0b08003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:53.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.654+0000 7f0b0effd640 1 -- 172.21.15.98:0/1150160251 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08046070 con 0x7f0b1015c470 2024-09-10T20:58:53.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.654+0000 7f0b0effd640 1 -- 172.21.15.98:0/1150160251 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b08004040 con 0x7f0b1015c470 2024-09-10T20:58:53.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.654+0000 7f0b182b8640 1 -- 172.21.15.98:0/1150160251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b101f5280 con 0x7f0b1015c470 2024-09-10T20:58:53.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.654+0000 7f0b0effd640 1 -- 172.21.15.98:0/1150160251 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08004450 con 0x7f0b1015c470 2024-09-10T20:58:53.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.654+0000 7f0b182b8640 1 -- 172.21.15.98:0/1150160251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b10142480 con 0x7f0b1015c470 2024-09-10T20:58:53.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.655+0000 7f0b182b8640 1 -- 172.21.15.98:0/1150160251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b10070050 con 0x7f0b1015c470 2024-09-10T20:58:53.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.656+0000 7f0b0effd640 1 -- 172.21.15.98:0/1150160251 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0b080045f0 con 0x7f0b1015c470 2024-09-10T20:58:53.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.656+0000 7f0b0effd640 1 --2- 172.21.15.98:0/1150160251 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0afc077f70 0x7f0afc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:53.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.656+0000 7f0b0effd640 1 -- 172.21.15.98:0/1150160251 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0b080bb130 con 0x7f0b1015c470 2024-09-10T20:58:53.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.656+0000 7f0b16054640 1 --2- 172.21.15.98:0/1150160251 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0afc077f70 0x7f0afc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:53.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.657+0000 7f0b16054640 1 --2- 172.21.15.98:0/1150160251 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0afc077f70 0x7f0afc07a430 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0b10144c10 tx=0x7f0b00005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:53.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.660+0000 7f0b0effd640 1 -- 172.21.15.98:0/1150160251 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0b08032030 con 0x7f0b1015c470 2024-09-10T20:58:53.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.836+0000 7f0b182b8640 1 -- 172.21.15.98:0/1150160251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b100cd6d0 con 0x7f0b1015c470 2024-09-10T20:58:53.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.837+0000 7f0b0effd640 1 -- 172.21.15.98:0/1150160251 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18610 (secure 0 0 0) 0x7f0b080832a0 con 0x7f0b1015c470 2024-09-10T20:58:53.841 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:58:53.841 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":25,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:58:51.602367+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:58:44.390290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T20:58:45.564099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T20:58:49.250508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:58:53.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.839+0000 7f0b182b8640 1 -- 172.21.15.98:0/1150160251 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0afc077f70 msgr2=0x7f0afc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.839+0000 7f0b182b8640 1 --2- 172.21.15.98:0/1150160251 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0afc077f70 0x7f0afc07a430 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0b10144c10 tx=0x7f0b00005f90 comp rx=0 tx=0).stop 2024-09-10T20:58:53.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.839+0000 7f0b182b8640 1 -- 172.21.15.98:0/1150160251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b1015c470 msgr2=0x7f0b101f48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:53.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.839+0000 7f0b182b8640 1 --2- 172.21.15.98:0/1150160251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b1015c470 0x7f0b101f48c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009bf0 tx=0x7f0b08003ba0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.839+0000 7f0b182b8640 1 -- 172.21.15.98:0/1150160251 shutdown_connections 2024-09-10T20:58:53.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.839+0000 7f0b182b8640 1 --2- 172.21.15.98:0/1150160251 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0afc077f70 0x7f0afc07a430 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.839+0000 7f0b182b8640 1 --2- 172.21.15.98:0/1150160251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b1015c470 0x7f0b101f48c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.839+0000 7f0b182b8640 1 --2- 172.21.15.98:0/1150160251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b10073780 0x7f0b100b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.839+0000 7f0b182b8640 1 --2- 172.21.15.98:0/1150160251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b1006f140 0x7f0b100ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:53.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.839+0000 7f0b182b8640 1 -- 172.21.15.98:0/1150160251 >> 172.21.15.98:0/1150160251 conn(0x7f0b100b1760 msgr2=0x7f0b100ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:53.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.839+0000 7f0b182b8640 1 -- 172.21.15.98:0/1150160251 shutdown_connections 2024-09-10T20:58:53.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:53.839+0000 7f0b182b8640 1 -- 172.21.15.98:0/1150160251 wait complete. 2024-09-10T20:58:54.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T20:58:46.560479+0000', 'modified': '2024-09-10T20:58:51.602516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4380}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4380': {'gid': 4380, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6835/3141703317', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 3141703317}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 3141703317}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4380, 'qdb_cluster': [4380]} 2024-09-10T20:58:54.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T20:58:54.258 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T20:58:54.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.358+0000 7fd042f17640 1 -- 172.21.15.98:0/2317057099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd02c0052e0 msgr2=0x7fd02c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:54.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.358+0000 7fd042f17640 1 --2- 172.21.15.98:0/2317057099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd02c0052e0 0x7fd02c00b9c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd03400c610 tx=0x7fd02c005830 comp rx=0 tx=0).stop 2024-09-10T20:58:54.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.358+0000 7fd042f17640 1 -- 172.21.15.98:0/2317057099 shutdown_connections 2024-09-10T20:58:54.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.358+0000 7fd042f17640 1 --2- 172.21.15.98:0/2317057099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd02c0052e0 0x7fd02c00b9c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.358+0000 7fd042f17640 1 -- 172.21.15.98:0/2317057099 >> 172.21.15.98:0/2317057099 conn(0x7fd03c0d1d10 msgr2=0x7fd03c0d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:54.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.358+0000 7fd042f17640 1 -- 172.21.15.98:0/2317057099 shutdown_connections 2024-09-10T20:58:54.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.358+0000 7fd042f17640 1 -- 172.21.15.98:0/2317057099 wait complete. 2024-09-10T20:58:54.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.359+0000 7fd042f17640 1 Processor -- start 2024-09-10T20:58:54.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.359+0000 7fd042f17640 1 -- start start 2024-09-10T20:58:54.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd042f17640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd02c0052e0 0x7fd03c14d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd042f17640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd03c0c8250 0x7fd03c14d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd042f17640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd03c0b57f0 0x7fd03c0b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd042f17640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c070af0 con 0x7fd02c0052e0 2024-09-10T20:58:54.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd042f17640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c070970 con 0x7fd03c0b57f0 2024-09-10T20:58:54.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd042f17640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd03c0cf8e0 con 0x7fd03c0c8250 2024-09-10T20:58:54.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd040cb3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd02c0052e0 0x7fd03c14d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd040cb3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd02c0052e0 0x7fd03c14d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51946/0 (socket says 172.21.15.98:51946) 2024-09-10T20:58:54.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd040cb3640 1 -- 172.21.15.98:0/597281314 learned_addr learned my addr 172.21.15.98:0/597281314 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:54.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd03bfff640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd03c0c8250 0x7fd03c14d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd0414b4640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd03c0b57f0 0x7fd03c0b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd040cb3640 1 -- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd03c0c8250 msgr2=0x7fd03c14d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:54.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd040cb3640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd03c0c8250 0x7fd03c14d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd040cb3640 1 -- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd03c0b57f0 msgr2=0x7fd03c0b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:54.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd040cb3640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd03c0b57f0 0x7fd03c0b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd040cb3640 1 -- 172.21.15.98:0/597281314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c0b4d90 con 0x7fd02c0052e0 2024-09-10T20:58:54.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd03bfff640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd03c0c8250 0x7fd03c14d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:58:54.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd0414b4640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd03c0b57f0 0x7fd03c0b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:58:54.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd040cb3640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd02c0052e0 0x7fd03c14d1d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd030003e40 tx=0x7fd030001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:54.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.360+0000 7fd0397fa640 1 -- 172.21.15.98:0/597281314 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030015020 con 0x7fd02c0052e0 2024-09-10T20:58:54.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.361+0000 7fd0397fa640 1 -- 172.21.15.98:0/597281314 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd030011840 con 0x7fd02c0052e0 2024-09-10T20:58:54.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.361+0000 7fd042f17640 1 -- 172.21.15.98:0/597281314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03c0b5070 con 0x7fd02c0052e0 2024-09-10T20:58:54.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.361+0000 7fd0397fa640 1 -- 172.21.15.98:0/597281314 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030011c50 con 0x7fd02c0052e0 2024-09-10T20:58:54.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.361+0000 7fd042f17640 1 -- 172.21.15.98:0/597281314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03c0cadc0 con 0x7fd02c0052e0 2024-09-10T20:58:54.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.362+0000 7fd0397fa640 1 -- 172.21.15.98:0/597281314 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd03001a400 con 0x7fd02c0052e0 2024-09-10T20:58:54.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.362+0000 7fd0397fa640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd020077f70 0x7fd02007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.363+0000 7fd0397fa640 1 -- 172.21.15.98:0/597281314 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd03009ad20 con 0x7fd02c0052e0 2024-09-10T20:58:54.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.363+0000 7fd03bfff640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd020077f70 0x7fd02007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.363+0000 7fd03bfff640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd020077f70 0x7fd02007a430 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd028005fd0 tx=0x7fd02800d570 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:54.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.364+0000 7fd045dc0740 1 Processor -- start 2024-09-10T20:58:54.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.364+0000 7fd045dc0740 1 -- start start 2024-09-10T20:58:54.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.364+0000 7fd045dc0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a71a1b0 0x55cc5a6f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.364+0000 7fd045dc0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a739df0 0x55cc5a73a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.364+0000 7fd045dc0740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a6ef5f0 0x55cc5a6efa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.364+0000 7fd045dc0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cc5a666820 con 0x55cc5a71a1b0 2024-09-10T20:58:54.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.364+0000 7fd045dc0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cc5a655c50 con 0x55cc5a739df0 2024-09-10T20:58:54.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.364+0000 7fd045dc0740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55cc5a659a60 con 0x55cc5a6ef5f0 2024-09-10T20:58:54.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.364+0000 7fd040cb3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a71a1b0 0x55cc5a6f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.364+0000 7fd040cb3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a71a1b0 0x55cc5a6f1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51954/0 (socket says 172.21.15.98:51954) 2024-09-10T20:58:54.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.364+0000 7fd040cb3640 1 -- 172.21.15.98:0/1236295426 learned_addr learned my addr 172.21.15.98:0/1236295426 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:54.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.365+0000 7fd040cb3640 1 -- 172.21.15.98:0/1236295426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a6ef5f0 msgr2=0x55cc5a6efa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:58:54.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.365+0000 7fd0414b4640 1 --2- 172.21.15.98:0/1236295426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a6ef5f0 0x55cc5a6efa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.365+0000 7fd03bfff640 1 --2- 172.21.15.98:0/1236295426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a739df0 0x55cc5a73a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.365+0000 7fd040cb3640 1 --2- 172.21.15.98:0/1236295426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a6ef5f0 0x55cc5a6efa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.365+0000 7fd040cb3640 1 -- 172.21.15.98:0/1236295426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a739df0 msgr2=0x55cc5a73a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:58:54.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.365+0000 7fd040cb3640 1 --2- 172.21.15.98:0/1236295426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a739df0 0x55cc5a73a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.365+0000 7fd040cb3640 1 -- 172.21.15.98:0/1236295426 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc5a3b7850 con 0x55cc5a71a1b0 2024-09-10T20:58:54.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.365+0000 7fd0414b4640 1 --2- 172.21.15.98:0/1236295426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a6ef5f0 0x55cc5a6efa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:58:54.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.365+0000 7fd03bfff640 1 --2- 172.21.15.98:0/1236295426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a739df0 0x55cc5a73a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:58:54.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.365+0000 7fd040cb3640 1 --2- 172.21.15.98:0/1236295426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a71a1b0 0x55cc5a6f1b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd030006ef0 tx=0x7fd0300625e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fdd9dfb7e32d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:54.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.365+0000 7fd0267fc640 1 -- 172.21.15.98:0/1236295426 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030088070 con 0x55cc5a71a1b0 2024-09-10T20:58:54.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.365+0000 7fd0267fc640 1 -- 172.21.15.98:0/1236295426 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd030062e70 con 0x55cc5a71a1b0 2024-09-10T20:58:54.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.365+0000 7fd0267fc640 1 -- 172.21.15.98:0/1236295426 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03004e390 con 0x55cc5a71a1b0 2024-09-10T20:58:54.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.366+0000 7fd045dc0740 1 -- 172.21.15.98:0/1236295426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a71a1b0 msgr2=0x55cc5a6f1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:54.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.366+0000 7fd045dc0740 1 --2- 172.21.15.98:0/1236295426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a71a1b0 0x55cc5a6f1b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd030006ef0 tx=0x7fd0300625e0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.366+0000 7fd045dc0740 1 -- 172.21.15.98:0/1236295426 shutdown_connections 2024-09-10T20:58:54.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.366+0000 7fd045dc0740 1 --2- 172.21.15.98:0/1236295426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a6ef5f0 0x55cc5a6efa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.366+0000 7fd045dc0740 1 --2- 172.21.15.98:0/1236295426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a739df0 0x55cc5a73a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.366+0000 7fd045dc0740 1 --2- 172.21.15.98:0/1236295426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a71a1b0 0x55cc5a6f1b20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.366+0000 7fd045dc0740 1 -- 172.21.15.98:0/1236295426 >> 172.21.15.98:0/1236295426 conn(0x55cc5a6f06a0 msgr2=0x55cc5a6f40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:54.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.366+0000 7fd045dc0740 1 -- 172.21.15.98:0/1236295426 shutdown_connections 2024-09-10T20:58:54.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.366+0000 7fd045dc0740 1 -- 172.21.15.98:0/1236295426 wait complete. 2024-09-10T20:58:54.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.366+0000 7fd045dc0740 1 -- start start 2024-09-10T20:58:54.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.366+0000 7fd045dc0740 1 Processor -- start 2024-09-10T20:58:54.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.367+0000 7fd045dc0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a71a1b0 0x55cc5a715520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.367+0000 7fd045dc0740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a739df0 0x55cc5a715a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.367+0000 7fd045dc0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a74d170 0x55cc5a71c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.367+0000 7fd045dc0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cc5a3c6620 con 0x55cc5a71a1b0 2024-09-10T20:58:54.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.367+0000 7fd045dc0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cc5a5fb5f0 con 0x55cc5a74d170 2024-09-10T20:58:54.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.367+0000 7fd045dc0740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55cc5a6eeb30 con 0x55cc5a739df0 2024-09-10T20:58:54.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.367+0000 7fd045dc0740 10 client.0 _ll_register_callbacks cb 0x55cc5a5ff0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T20:58:54.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.367+0000 7fd040cb3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a71a1b0 0x55cc5a715520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.367+0000 7fd040cb3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a71a1b0 0x55cc5a715520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51958/0 (socket says 172.21.15.98:51958) 2024-09-10T20:58:54.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.367+0000 7fd040cb3640 1 -- 172.21.15.98:0/2949458138 learned_addr learned my addr 172.21.15.98:0/2949458138 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:54.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.367+0000 7fd03bfff640 1 --2- 172.21.15.98:0/2949458138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a739df0 0x55cc5a715a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.367+0000 7fd0414b4640 1 --2- 172.21.15.98:0/2949458138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a74d170 0x55cc5a71c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd040cb3640 1 -- 172.21.15.98:0/2949458138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a739df0 msgr2=0x55cc5a715a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:58:54.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd040cb3640 1 --2- 172.21.15.98:0/2949458138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a739df0 0x55cc5a715a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd040cb3640 1 -- 172.21.15.98:0/2949458138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a74d170 msgr2=0x55cc5a71c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:58:54.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd040cb3640 1 --2- 172.21.15.98:0/2949458138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a74d170 0x55cc5a71c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd040cb3640 1 -- 172.21.15.98:0/2949458138 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc5a42e6b0 con 0x55cc5a71a1b0 2024-09-10T20:58:54.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd0414b4640 1 --2- 172.21.15.98:0/2949458138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a74d170 0x55cc5a71c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:58:54.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd040cb3640 1 --2- 172.21.15.98:0/2949458138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a71a1b0 0x55cc5a715520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd030001b30 tx=0x7fd030001e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426ef989fdc70aa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:54.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd01b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-10T20:58:54.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd01b7fe640 1 -- 172.21.15.98:0/2949458138 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd030088040 con 0x55cc5a71a1b0 2024-09-10T20:58:54.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd01b7fe640 1 -- 172.21.15.98:0/2949458138 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd0300622e0 con 0x55cc5a71a1b0 2024-09-10T20:58:54.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd045dc0740 1 -- 172.21.15.98:0/2949458138 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd030062e70 con 0x55cc5a71a1b0 2024-09-10T20:58:54.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd01b7fe640 1 -- 172.21.15.98:0/2949458138 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fd03006c370 con 0x55cc5a71a1b0 2024-09-10T20:58:54.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd045dc0740 10 client.4509 fetch_fsmap learned FSMap version 6 2024-09-10T20:58:54.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd045dc0740 1 -- 172.21.15.98:0/2949458138 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55cc5a713720 con 0x55cc5a71a1b0 2024-09-10T20:58:54.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd01b7fe640 1 -- 172.21.15.98:0/2949458138 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd030062e70 con 0x55cc5a71a1b0 2024-09-10T20:58:54.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd01b7fe640 1 -- 172.21.15.98:0/2949458138 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 6) ==== 3510+0+0 (secure 0 0 0) 0x7fd030068ae0 con 0x55cc5a71a1b0 2024-09-10T20:58:54.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd045dc0740 10 client.4509 fetch_fsmap finished waiting for FSMap version 6 2024-09-10T20:58:54.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd045dc0740 10 client.4509 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T20:58:54.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd045dc0740 20 client.4509 populate_metadata read hostname 'smithi098' 2024-09-10T20:58:54.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd045dc0740 1 --2- 172.21.15.98:0/2949458138 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x55cc5a754a60 0x55cc5a756e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd045dc0740 4 client.4509 mds_command: new command op to 4380 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T20:58:54.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd045dc0740 1 -- 172.21.15.98:0/2949458138 --> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc5a668770 con 0x55cc5a754a60 2024-09-10T20:58:54.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.368+0000 7fd03bfff640 1 --2- 172.21.15.98:0/2949458138 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x55cc5a754a60 0x55cc5a756e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.369+0000 7fd01b7fe640 10 client.4509 ms_handle_connect on v2:172.21.15.98:6834/3141703317 2024-09-10T20:58:54.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.369+0000 7fd03bfff640 1 --2- 172.21.15.98:0/2949458138 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x55cc5a754a60 0x55cc5a756e50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de806eec09e5c5b server_cookie=8806d4dad0e8b46 in_seq=0 out_seq=0 2024-09-10T20:58:54.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.369+0000 7fd01b7fe640 1 -- 172.21.15.98:0/2949458138 <== mds.0 v2:172.21.15.98:6834/3141703317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cc5a668770 con 0x55cc5a754a60 2024-09-10T20:58:54.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.369+0000 7fd01b7fe640 10 client.4509 handle_command_reply: tid=0 2024-09-10T20:58:54.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.369+0000 7fd045dc0740 1 client.4509 shutdown 2024-09-10T20:58:54.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.369+0000 7fd045dc0740 1 -- 172.21.15.98:0/2949458138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a71a1b0 msgr2=0x55cc5a715520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:54.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.369+0000 7fd045dc0740 1 --2- 172.21.15.98:0/2949458138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a71a1b0 0x55cc5a715520 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd030001b30 tx=0x7fd030001e00 comp rx=0 tx=0).stop 2024-09-10T20:58:54.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.370+0000 7fd045dc0740 1 -- 172.21.15.98:0/2949458138 shutdown_connections 2024-09-10T20:58:54.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.370+0000 7fd045dc0740 1 --2- 172.21.15.98:0/2949458138 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x55cc5a754a60 0x55cc5a756e50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.370+0000 7fd045dc0740 1 --2- 172.21.15.98:0/2949458138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a74d170 0x55cc5a71c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.370+0000 7fd045dc0740 1 --2- 172.21.15.98:0/2949458138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a739df0 0x55cc5a715a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.370+0000 7fd045dc0740 1 --2- 172.21.15.98:0/2949458138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a71a1b0 0x55cc5a715520 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.370+0000 7fd01b7fe640 0 client.4509 ms_handle_reset on v2:172.21.15.98:6834/3141703317 2024-09-10T20:58:54.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.370+0000 7fd045dc0740 1 -- 172.21.15.98:0/2949458138 >> 172.21.15.98:0/2949458138 conn(0x55cc5a6ef5f0 msgr2=0x55cc5a71a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:54.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.370+0000 7fd045dc0740 1 -- 172.21.15.98:0/2949458138 shutdown_connections 2024-09-10T20:58:54.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.370+0000 7fd045dc0740 1 -- 172.21.15.98:0/2949458138 wait complete. 2024-09-10T20:58:54.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.370+0000 7fd045dc0740 20 client.4509 trim_cache size 0 max 16384 2024-09-10T20:58:54.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.388+0000 7fd045dc0740 1 Processor -- start 2024-09-10T20:58:54.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.388+0000 7fd045dc0740 1 -- start start 2024-09-10T20:58:54.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd045dc0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a71a1b0 0x55cc5a7b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd045dc0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a739df0 0x55cc5a6ed750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd045dc0740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a74d170 0x55cc5a6edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd045dc0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c1ffa80 con 0x55cc5a739df0 2024-09-10T20:58:54.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd045dc0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c0c9960 con 0x55cc5a71a1b0 2024-09-10T20:58:54.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd045dc0740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd03c1ff440 con 0x55cc5a74d170 2024-09-10T20:58:54.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd03bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a739df0 0x55cc5a6ed750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd03bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a739df0 0x55cc5a6ed750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51964/0 (socket says 172.21.15.98:51964) 2024-09-10T20:58:54.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd03bfff640 1 -- 172.21.15.98:0/1330173272 learned_addr learned my addr 172.21.15.98:0/1330173272 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:54.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd03bfff640 1 -- 172.21.15.98:0/1330173272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a74d170 msgr2=0x55cc5a6edc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:58:54.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd03bfff640 1 --2- 172.21.15.98:0/1330173272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a74d170 0x55cc5a6edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd03bfff640 1 -- 172.21.15.98:0/1330173272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a71a1b0 msgr2=0x55cc5a7b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:58:54.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd03bfff640 1 --2- 172.21.15.98:0/1330173272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a71a1b0 0x55cc5a7b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd03bfff640 1 -- 172.21.15.98:0/1330173272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc5a713720 con 0x55cc5a739df0 2024-09-10T20:58:54.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd03bfff640 1 --2- 172.21.15.98:0/1330173272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a739df0 0x55cc5a6ed750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd0280127a0 tx=0x7fd028014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d270e14a98099c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:54.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd025ffb640 1 -- 172.21.15.98:0/1330173272 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028011dd0 con 0x55cc5a739df0 2024-09-10T20:58:54.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd025ffb640 1 -- 172.21.15.98:0/1330173272 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd028012d90 con 0x55cc5a739df0 2024-09-10T20:58:54.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.389+0000 7fd025ffb640 1 -- 172.21.15.98:0/1330173272 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028031630 con 0x55cc5a739df0 2024-09-10T20:58:54.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.390+0000 7fd045dc0740 1 -- 172.21.15.98:0/1330173272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a739df0 msgr2=0x55cc5a6ed750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:54.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.390+0000 7fd045dc0740 1 --2- 172.21.15.98:0/1330173272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a739df0 0x55cc5a6ed750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd0280127a0 tx=0x7fd028014000 comp rx=0 tx=0).stop 2024-09-10T20:58:54.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.390+0000 7fd045dc0740 1 -- 172.21.15.98:0/1330173272 shutdown_connections 2024-09-10T20:58:54.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.390+0000 7fd045dc0740 1 --2- 172.21.15.98:0/1330173272 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a74d170 0x55cc5a6edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.390+0000 7fd045dc0740 1 --2- 172.21.15.98:0/1330173272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a739df0 0x55cc5a6ed750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.390+0000 7fd045dc0740 1 --2- 172.21.15.98:0/1330173272 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a71a1b0 0x55cc5a7b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.390+0000 7fd045dc0740 1 -- 172.21.15.98:0/1330173272 >> 172.21.15.98:0/1330173272 conn(0x55cc5a6ef5f0 msgr2=0x55cc5a716020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:54.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.390+0000 7fd045dc0740 1 -- 172.21.15.98:0/1330173272 shutdown_connections 2024-09-10T20:58:54.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.390+0000 7fd045dc0740 1 -- 172.21.15.98:0/1330173272 wait complete. 2024-09-10T20:58:54.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.390+0000 7fd045dc0740 1 -- start start 2024-09-10T20:58:54.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.391+0000 7fd045dc0740 1 Processor -- start 2024-09-10T20:58:54.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.391+0000 7fd045dc0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a71a1b0 0x55cc5a713b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.391+0000 7fd045dc0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a739df0 0x55cc5a7b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.391+0000 7fd045dc0740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a74d170 0x55cc5a7b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.391+0000 7fd045dc0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cc5a7133f0 con 0x55cc5a739df0 2024-09-10T20:58:54.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.391+0000 7fd045dc0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55cc5a3c6620 con 0x55cc5a71a1b0 2024-09-10T20:58:54.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.391+0000 7fd045dc0740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55cc5a749b60 con 0x55cc5a74d170 2024-09-10T20:58:54.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.391+0000 7fd045dc0740 10 client.0 _ll_register_callbacks cb 0x7fd03004d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T20:58:54.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.391+0000 7fd03bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a739df0 0x55cc5a7b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.391+0000 7fd03bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a739df0 0x55cc5a7b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51972/0 (socket says 172.21.15.98:51972) 2024-09-10T20:58:54.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.391+0000 7fd03bfff640 1 -- 172.21.15.98:0/2692023069 learned_addr learned my addr 172.21.15.98:0/2692023069 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:54.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd040cb3640 1 --2- 172.21.15.98:0/2692023069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a71a1b0 0x55cc5a713b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd0414b4640 1 --2- 172.21.15.98:0/2692023069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a74d170 0x55cc5a7b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd03bfff640 1 -- 172.21.15.98:0/2692023069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a74d170 msgr2=0x55cc5a7b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:58:54.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd03bfff640 1 --2- 172.21.15.98:0/2692023069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a74d170 0x55cc5a7b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd03bfff640 1 -- 172.21.15.98:0/2692023069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a71a1b0 msgr2=0x55cc5a713b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:58:54.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd03bfff640 1 --2- 172.21.15.98:0/2692023069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a71a1b0 0x55cc5a713b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd03bfff640 1 -- 172.21.15.98:0/2692023069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc5a7abd70 con 0x55cc5a739df0 2024-09-10T20:58:54.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd040cb3640 1 --2- 172.21.15.98:0/2692023069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a71a1b0 0x55cc5a713b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:58:54.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd03bfff640 1 --2- 172.21.15.98:0/2692023069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a739df0 0x55cc5a7b2950 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd03c155d00 tx=0x7fd028011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee13a7d760959123 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:54.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd01b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-10T20:58:54.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd01b7fe640 1 -- 172.21.15.98:0/2692023069 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028031740 con 0x55cc5a739df0 2024-09-10T20:58:54.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd01b7fe640 1 -- 172.21.15.98:0/2692023069 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd028036040 con 0x55cc5a739df0 2024-09-10T20:58:54.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd045dc0740 1 -- 172.21.15.98:0/2692023069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd028012d90 con 0x55cc5a739df0 2024-09-10T20:58:54.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd01b7fe640 1 -- 172.21.15.98:0/2692023069 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fd0280318e0 con 0x55cc5a739df0 2024-09-10T20:58:54.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd045dc0740 10 client.4515 fetch_fsmap learned FSMap version 6 2024-09-10T20:58:54.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd045dc0740 1 -- 172.21.15.98:0/2692023069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55cc5a7ab820 con 0x55cc5a739df0 2024-09-10T20:58:54.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd01b7fe640 1 -- 172.21.15.98:0/2692023069 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd028012d90 con 0x55cc5a739df0 2024-09-10T20:58:54.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd01b7fe640 1 -- 172.21.15.98:0/2692023069 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 6) ==== 3510+0+0 (secure 0 0 0) 0x7fd02803a070 con 0x55cc5a739df0 2024-09-10T20:58:54.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd045dc0740 10 client.4515 fetch_fsmap finished waiting for FSMap version 6 2024-09-10T20:58:54.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd045dc0740 10 client.4515 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T20:58:54.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd045dc0740 20 client.4515 populate_metadata read hostname 'smithi098' 2024-09-10T20:58:54.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd045dc0740 1 --2- 172.21.15.98:0/2692023069 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x55cc5a7be360 0x55cc5a7be740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd045dc0740 4 client.4515 mds_command: new command op to 4380 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T20:58:54.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.392+0000 7fd045dc0740 1 -- 172.21.15.98:0/2692023069 --> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] -- command(tid 0: {"prefix": "status"}) -- 0x55cc5a733b60 con 0x55cc5a7be360 2024-09-10T20:58:54.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd040cb3640 1 --2- 172.21.15.98:0/2692023069 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x55cc5a7be360 0x55cc5a7be740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd01b7fe640 10 client.4515 ms_handle_connect on v2:172.21.15.98:6834/3141703317 2024-09-10T20:58:54.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd040cb3640 1 --2- 172.21.15.98:0/2692023069 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x55cc5a7be360 0x55cc5a7be740 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=750227306caee5c9 server_cookie=a77b942aa68d4cc5 in_seq=0 out_seq=0 2024-09-10T20:58:54.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd01b7fe640 1 -- 172.21.15.98:0/2692023069 <== mds.0 v2:172.21.15.98:6834/3141703317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55cc5a733b60 con 0x55cc5a7be360 2024-09-10T20:58:54.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd01b7fe640 10 client.4515 handle_command_reply: tid=0 2024-09-10T20:58:54.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd045dc0740 1 client.4515 shutdown 2024-09-10T20:58:54.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd045dc0740 1 -- 172.21.15.98:0/2692023069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a739df0 msgr2=0x55cc5a7b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:54.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd045dc0740 1 --2- 172.21.15.98:0/2692023069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a739df0 0x55cc5a7b2950 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd03c155d00 tx=0x7fd028011f20 comp rx=0 tx=0).stop 2024-09-10T20:58:54.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd045dc0740 1 -- 172.21.15.98:0/2692023069 shutdown_connections 2024-09-10T20:58:54.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd045dc0740 1 --2- 172.21.15.98:0/2692023069 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x55cc5a7be360 0x55cc5a7be740 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd045dc0740 1 --2- 172.21.15.98:0/2692023069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cc5a74d170 0x55cc5a7b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd01b7fe640 0 client.4515 ms_handle_reset on v2:172.21.15.98:6834/3141703317 2024-09-10T20:58:54.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd045dc0740 1 --2- 172.21.15.98:0/2692023069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cc5a739df0 0x55cc5a7b2950 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd045dc0740 1 --2- 172.21.15.98:0/2692023069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cc5a71a1b0 0x55cc5a713b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd045dc0740 1 -- 172.21.15.98:0/2692023069 >> 172.21.15.98:0/2692023069 conn(0x55cc5a6ef5f0 msgr2=0x55cc5a71a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:54.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd045dc0740 1 -- 172.21.15.98:0/2692023069 shutdown_connections 2024-09-10T20:58:54.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd045dc0740 1 -- 172.21.15.98:0/2692023069 wait complete. 2024-09-10T20:58:54.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.393+0000 7fd045dc0740 20 client.4515 trim_cache size 0 max 16384 2024-09-10T20:58:54.404 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-10T20:58:54.404 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "41041377-a016-49fd-be1f-323d0b834557", 2024-09-10T20:58:54.405 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-10T20:58:54.405 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 4380, 2024-09-10T20:58:54.405 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-10T20:58:54.405 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-10T20:58:54.405 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-10T20:58:54.405 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 7.1763290910000004, 2024-09-10T20:58:54.405 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 6, 2024-09-10T20:58:54.405 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 25, 2024-09-10T20:58:54.405 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 24, 2024-09-10T20:58:54.405 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 13.205541574 2024-09-10T20:58:54.405 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-10T20:58:54.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.394+0000 7fd042f17640 1 -- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd020077f70 msgr2=0x7fd02007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:54.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.394+0000 7fd042f17640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd020077f70 0x7fd02007a430 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd028005fd0 tx=0x7fd02800d570 comp rx=0 tx=0).stop 2024-09-10T20:58:54.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.394+0000 7fd042f17640 1 -- 172.21.15.98:0/597281314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd02c0052e0 msgr2=0x7fd03c14d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:54.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.394+0000 7fd042f17640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd02c0052e0 0x7fd03c14d1d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd030003e40 tx=0x7fd030001630 comp rx=0 tx=0).stop 2024-09-10T20:58:54.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.394+0000 7fd042f17640 1 -- 172.21.15.98:0/597281314 shutdown_connections 2024-09-10T20:58:54.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.394+0000 7fd042f17640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd020077f70 0x7fd02007a430 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.394+0000 7fd042f17640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd03c0b57f0 0x7fd03c0b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.394+0000 7fd042f17640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd03c0c8250 0x7fd03c14d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.394+0000 7fd042f17640 1 --2- 172.21.15.98:0/597281314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd02c0052e0 0x7fd03c14d1d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.394+0000 7fd042f17640 1 -- 172.21.15.98:0/597281314 >> 172.21.15.98:0/597281314 conn(0x7fd03c0d1d10 msgr2=0x7fd03c0b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:54.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.394+0000 7fd042f17640 1 -- 172.21.15.98:0/597281314 shutdown_connections 2024-09-10T20:58:54.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.394+0000 7fd042f17640 1 -- 172.21.15.98:0/597281314 wait complete. 2024-09-10T20:58:54.408 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T20:58:54.409 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T20:58:54.409 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T20:58:54.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.506+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/2396269309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa2c004d40 msgr2=0x7ffa2c0051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:54.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.506+0000 7ffa4a2c6640 1 --2- 172.21.15.98:0/2396269309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa2c004d40 0x7ffa2c0051a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009ed0 tx=0x7ffa2c005830 comp rx=0 tx=0).stop 2024-09-10T20:58:54.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.506+0000 7ffa48863640 1 -- 172.21.15.98:0/2396269309 reap_dead start 2024-09-10T20:58:54.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.506+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/2396269309 shutdown_connections 2024-09-10T20:58:54.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.506+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/2396269309 >> 172.21.15.98:0/2396269309 conn(0x7ffa440d3f70 msgr2=0x7ffa440d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:54.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.506+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/2396269309 shutdown_connections 2024-09-10T20:58:54.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.506+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/2396269309 wait complete. 2024-09-10T20:58:54.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.507+0000 7ffa4a2c6640 1 Processor -- start 2024-09-10T20:58:54.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.507+0000 7ffa4a2c6640 1 -- start start 2024-09-10T20:58:54.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa4a2c6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c004d40 0x7ffa441612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa4a2c6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa4415a840 0x7ffa4415edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa4a2c6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa4415a2d0 0x7ffa4406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa4a2c6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44166f70 con 0x7ffa4415a840 2024-09-10T20:58:54.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa4a2c6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44166df0 con 0x7ffa4415a2d0 2024-09-10T20:58:54.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa4a2c6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffa441670f0 con 0x7ffa2c004d40 2024-09-10T20:58:54.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa437fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa4415a840 0x7ffa4415edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa437fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa4415a840 0x7ffa4415edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51984/0 (socket says 172.21.15.98:51984) 2024-09-10T20:58:54.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa437fe640 1 -- 172.21.15.98:0/3562985803 learned_addr learned my addr 172.21.15.98:0/3562985803 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:54.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa48863640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa4415a2d0 0x7ffa4406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa43fff640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c004d40 0x7ffa441612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa437fe640 1 -- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c004d40 msgr2=0x7ffa441612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:54.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa437fe640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c004d40 0x7ffa441612e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa437fe640 1 -- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa4415a2d0 msgr2=0x7ffa4406d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:54.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa437fe640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa4415a2d0 0x7ffa4406d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa437fe640 1 -- 172.21.15.98:0/3562985803 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4415eaa0 con 0x7ffa4415a840 2024-09-10T20:58:54.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa43fff640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c004d40 0x7ffa441612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:58:54.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa437fe640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa4415a840 0x7ffa4415edd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800e5e0 tx=0x7ffa3800eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:54.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.508+0000 7ffa40ff9640 1 -- 172.21.15.98:0/3562985803 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38015020 con 0x7ffa4415a840 2024-09-10T20:58:54.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.509+0000 7ffa40ff9640 1 -- 172.21.15.98:0/3562985803 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa3800ca00 con 0x7ffa4415a840 2024-09-10T20:58:54.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.509+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/3562985803 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4415e4e0 con 0x7ffa4415a840 2024-09-10T20:58:54.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.509+0000 7ffa40ff9640 1 -- 172.21.15.98:0/3562985803 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800ce30 con 0x7ffa4415a840 2024-09-10T20:58:54.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.509+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/3562985803 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa44160ae0 con 0x7ffa4415a840 2024-09-10T20:58:54.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.509+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/3562985803 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa4408dfa0 con 0x7ffa4415a840 2024-09-10T20:58:54.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.510+0000 7ffa40ff9640 1 -- 172.21.15.98:0/3562985803 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ffa38011460 con 0x7ffa4415a840 2024-09-10T20:58:54.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.511+0000 7ffa40ff9640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa30077f70 0x7ffa3007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:54.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.511+0000 7ffa40ff9640 1 -- 172.21.15.98:0/3562985803 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffa3809a910 con 0x7ffa4415a840 2024-09-10T20:58:54.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.511+0000 7ffa43fff640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa30077f70 0x7ffa3007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:54.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.511+0000 7ffa43fff640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa30077f70 0x7ffa3007a430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffa34000c00 tx=0x7ffa340049d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:54.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.515+0000 7ffa40ff9640 1 -- 172.21.15.98:0/3562985803 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ffa3801f020 con 0x7ffa4415a840 2024-09-10T20:58:54.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.731+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/3562985803 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffa44161b00 con 0x7ffa4415a840 2024-09-10T20:58:54.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.731+0000 7ffa40ff9640 1 -- 172.21.15.98:0/3562985803 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7ffa380630d0 con 0x7ffa4415a840 2024-09-10T20:58:54.735 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T20:58:54.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.733+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa30077f70 msgr2=0x7ffa3007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:54.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.733+0000 7ffa4a2c6640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa30077f70 0x7ffa3007a430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffa34000c00 tx=0x7ffa340049d0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.733+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/3562985803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa4415a840 msgr2=0x7ffa4415edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:54.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.733+0000 7ffa4a2c6640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa4415a840 0x7ffa4415edd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800e5e0 tx=0x7ffa3800eaa0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.733+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/3562985803 shutdown_connections 2024-09-10T20:58:54.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.733+0000 7ffa4a2c6640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa30077f70 0x7ffa3007a430 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.733+0000 7ffa4a2c6640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa4415a2d0 0x7ffa4406d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.733+0000 7ffa4a2c6640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa4415a840 0x7ffa4415edd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.733+0000 7ffa4a2c6640 1 --2- 172.21.15.98:0/3562985803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa2c004d40 0x7ffa441612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:54.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.733+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/3562985803 >> 172.21.15.98:0/3562985803 conn(0x7ffa440d3f70 msgr2=0x7ffa4406ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:54.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.733+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/3562985803 shutdown_connections 2024-09-10T20:58:54.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54.733+0000 7ffa4a2c6640 1 -- 172.21.15.98:0/3562985803 wait complete. 2024-09-10T20:58:54.748 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T20:58:54.748 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T20:58:54.748 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-10T20:58:54.748 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-10T20:58:54.748 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T20:58:54.748 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T20:58:54.748 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T20:58:54.748 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T20:58:54.748 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T20:58:54.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:58:54.749 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-10T20:58:54.764 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T20:58:54.764 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T20:58:54.764 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T20:58:54.764 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:58:54.765 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-10T20:58:54.765 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:58:54.765 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:58:54.765 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:58:54.765 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-10T20:58:54.765 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:58:54.765 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:58:54.765 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-10T20:58:54.765 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:58:54.765 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-10T20:58:54.765 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T20:58:54.765 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-10T20:58:54.766 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T20:58:54.766 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42789sec preferred_lft 42789sec 2024-09-10T20:58:54.766 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link noprefixroute 2024-09-10T20:58:54.766 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:58:54.766 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T20:58:54.767 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:58:54.767 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:58:54.767 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add name ceph-brx type bridge 2024-09-10T20:58:54.767 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip addr flush dev ceph-brx 2024-09-10T20:58:54.767 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set ceph-brx up 2024-09-10T20:58:54.767 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T20:58:54.767 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:58:54.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:58:54.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:58:54.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:58:54.961 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T20:58:54.985 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-10T20:58:54.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:58:54.986 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-10T20:58:55.050 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-10T20:58:55.050 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T20:58:55.050 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T20:58:55.050 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T20:58:55.051 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T20:58:55.051 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:58:55.051 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:58:55.052 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T20:58:55.052 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T20:58:55.052 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T20:58:55.052 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:58:55.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:58:55.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:58:55.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:58:55.315 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-10T20:58:55.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:58:55.330 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list-id 2024-09-10T20:58:55.386 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:58:55.386 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:58:55.386 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T20:58:55.386 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-10T20:58:55.386 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:58:55.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:58:55.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:58:55.504 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-10T20:58:55.505 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:58:55.505 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:58:55.505 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-10T20:58:55.505 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T20:58:55.505 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-10T20:58:55.505 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-10T20:58:55.505 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-10T20:58:55.505 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:58:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:58:55.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:58:55.647 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:58:55.647 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:58:55.647 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-10T20:58:55.647 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T20:58:55.647 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:58:55.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:58:55.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:58:55.724 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-10T20:58:55.724 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T20:58:55.724 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-10T20:58:55.740 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T20:58:55.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T20:58:55.740 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-10T20:58:55.794 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-10T20:58:55.861 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T20:58:55.917 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-10T20:58:55.917 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-10T20:58:55.917 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-10T20:58:55.917 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-10T20:58:55.917 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-10T20:58:55.918 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-10T20:58:55.918 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-10T20:58:55.918 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-10T20:58:55.918 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-10T20:58:55.918 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-10T20:58:55.918 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-10T20:58:55.918 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/selinux 2024-09-10T20:58:55.918 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T20:58:55.918 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-10T20:58:55.918 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-10T20:58:55.918 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-10T20:58:55.918 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-10T20:58:55.918 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T20:58:55.918 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-10T20:58:55.919 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T20:58:55.919 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-10T20:58:55.919 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T20:58:55.919 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T20:58:55.919 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-10T20:58:55.919 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-10T20:58:55.919 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T20:58:55.919 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T20:58:55.919 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T20:58:55.919 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T20:58:55.919 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-10T20:58:55.919 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T20:58:55.919 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T20:58:55.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:58:55.920 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-10T20:58:55.976 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T20:58:55.977 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-10T20:58:56.019 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-10T20:58:56.044 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T20:58:56.101 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-10T20:58:56.101 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-10T20:58:56.101 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-10T20:58:56.101 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-10T20:58:56.101 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-10T20:58:56.101 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-10T20:58:56.102 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-10T20:58:56.102 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-10T20:58:56.102 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-10T20:58:56.102 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-10T20:58:56.102 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-10T20:58:56.102 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/selinux 2024-09-10T20:58:56.102 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T20:58:56.102 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-10T20:58:56.102 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-10T20:58:56.102 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-10T20:58:56.102 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-10T20:58:56.102 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T20:58:56.103 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-10T20:58:56.103 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T20:58:56.103 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-10T20:58:56.103 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T20:58:56.103 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T20:58:56.103 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-10T20:58:56.103 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-10T20:58:56.103 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T20:58:56.103 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T20:58:56.103 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T20:58:56.103 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T20:58:56.103 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-10T20:58:56.103 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T20:58:56.104 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T20:58:56.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:58:56.104 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-10T20:58:56.140 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi098.stderr:2024-09-10T20:58:56.136+0000 7f016b382480 -1 init, newargv = 0x563ee3aabdc0 newargc=15 2024-09-10T20:58:56.140 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi098.stderr:ceph-fuse[43543]: starting ceph client 2024-09-10T20:58:56.147 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi098.stderr:ceph-fuse[43543]: starting fuse 2024-09-10T20:58:56.156 INFO:teuthology.orchestra.run.smithi098.stdout:41 2024-09-10T20:58:56.157 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-10T20:58:56.157 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-10T20:58:56.157 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2024-09-10T20:58:56.157 DEBUG:teuthology.orchestra.run.smithi098:> import re 2024-09-10T20:58:56.157 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-10T20:58:56.157 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T20:58:56.158 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T20:58:56.159 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2024-09-10T20:58:56.159 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2024-09-10T20:58:56.159 DEBUG:teuthology.orchestra.run.smithi098:> return f 2024-09-10T20:58:56.159 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T20:58:56.159 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-10T20:58:56.159 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.admin")) 2024-09-10T20:58:56.159 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-10T20:58:56.273 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.admin.43543.asok 2024-09-10T20:58:56.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:58:56.284 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.43543.asok 2024-09-10T20:58:56.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:58:56.284 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.43543.asok status 2024-09-10T20:58:56.419 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-10T20:58:56.419 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2024-09-10T20:58:56.419 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T20:58:56.419 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T20:58:56.419 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "admin", 2024-09-10T20:58:56.419 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2024-09-10T20:58:56.419 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-10T20:58:56.419 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "43543", 2024-09-10T20:58:56.419 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2024-09-10T20:58:56.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:58:56.420 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2024-09-10T20:58:56.420 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2024-09-10T20:58:56.420 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 4527, 2024-09-10T20:58:56.420 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2024-09-10T20:58:56.420 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2024-09-10T20:58:56.420 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2024-09-10T20:58:56.420 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 4527 2024-09-10T20:58:56.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:58:56.420 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-10T20:58:56.420 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-10T20:58:56.420 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-10T20:58:56.420 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 541204735 2024-09-10T20:58:56.421 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:58:56.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:58:56.421 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-10T20:58:56.421 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-10T20:58:56.421 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-10T20:58:56.421 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 541204735 2024-09-10T20:58:56.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:58:56.421 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.4527 192.168.0.1:0/541204735", 2024-09-10T20:58:56.421 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.1:0/541204735", 2024-09-10T20:58:56.421 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2024-09-10T20:58:56.421 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 6, 2024-09-10T20:58:56.421 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 25, 2024-09-10T20:58:56.421 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2024-09-10T20:58:56.421 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2024-09-10T20:58:56.422 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2024-09-10T20:58:56.422 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-10T20:58:56.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:58:56.428 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2024-09-10T20:58:56.428 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-10T20:58:56.443 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2024-09-10T20:58:56.443 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-10T20:58:56.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:58:56.443 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-10T20:58:56.509 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-10T20:58:56.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T20:58:56.617 INFO:teuthology.orchestra.run.smithi098.stdout:# file: . 2024-09-10T20:58:56.617 INFO:teuthology.orchestra.run.smithi098.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-10T20:58:56.617 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:58:56.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:58:56.624 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:56.624 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:58:56.640 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T20:58:56.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.778+0000 7f9f82790640 1 -- 172.21.15.98:0/1921415830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7c06f140 msgr2=0x7f9f7c06f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:56.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.778+0000 7f9f82790640 1 --2- 172.21.15.98:0/1921415830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7c06f140 0x7f9f7c06f5b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c076000 tx=0x7f9f7402a8b0 comp rx=0 tx=0).stop 2024-09-10T20:58:56.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.778+0000 7f9f82790640 1 -- 172.21.15.98:0/1921415830 shutdown_connections 2024-09-10T20:58:56.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.778+0000 7f9f82790640 1 --2- 172.21.15.98:0/1921415830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7c06f140 0x7f9f7c06f5b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:56.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.778+0000 7f9f82790640 1 --2- 172.21.15.98:0/1921415830 >> v2:172.21.15.190:3300/0 conn(0x7f9f7c073780 0x7f9f7c073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:56.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.778+0000 7f9f82790640 1 -- 172.21.15.98:0/1921415830 >> 172.21.15.98:0/1921415830 conn(0x7f9f7c0b1760 msgr2=0x7f9f7c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:56.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.778+0000 7f9f82790640 1 -- 172.21.15.98:0/1921415830 shutdown_connections 2024-09-10T20:58:56.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.778+0000 7f9f82790640 1 -- 172.21.15.98:0/1921415830 wait complete. 2024-09-10T20:58:56.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.779+0000 7f9f82790640 1 Processor -- start 2024-09-10T20:58:56.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.779+0000 7f9f82790640 1 -- start start 2024-09-10T20:58:56.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f82790640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c06f140 0x7f9f7c140c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:56.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f82790640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c073780 0x7f9f7c0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:56.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f82790640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7c15c470 0x7f9f7c0aba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:56.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f82790640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c142260 con 0x7f9f7c15c470 2024-09-10T20:58:56.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f82790640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c1420e0 con 0x7f9f7c073780 2024-09-10T20:58:56.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f82790640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7c06faf0 con 0x7f9f7c06f140 2024-09-10T20:58:56.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f80d2d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7c15c470 0x7f9f7c0aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:56.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f80d2d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7c15c470 0x7f9f7c0aba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:51990/0 (socket says 172.21.15.98:51990) 2024-09-10T20:58:56.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f80d2d640 1 -- 172.21.15.98:0/2213993850 learned_addr learned my addr 172.21.15.98:0/2213993850 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:56.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f7bfff640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c06f140 0x7f9f7c140c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:56.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f7b7fe640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c073780 0x7f9f7c0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:56.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f80d2d640 1 -- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c06f140 msgr2=0x7f9f7c140c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:56.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f80d2d640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c06f140 0x7f9f7c140c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:56.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f80d2d640 1 -- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c073780 msgr2=0x7f9f7c0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:56.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f80d2d640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c073780 0x7f9f7c0aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:56.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.780+0000 7f9f80d2d640 1 -- 172.21.15.98:0/2213993850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7c1f3260 con 0x7f9f7c15c470 2024-09-10T20:58:56.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.781+0000 7f9f7bfff640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c06f140 0x7f9f7c140c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:58:56.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.781+0000 7f9f7b7fe640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c073780 0x7f9f7c0aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:58:56.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.781+0000 7f9f80d2d640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7c15c470 0x7f9f7c0aba80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009ca0 tx=0x7f9f74003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:56.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.781+0000 7f9f78ff9640 1 -- 172.21.15.98:0/2213993850 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74040070 con 0x7f9f7c15c470 2024-09-10T20:58:56.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.781+0000 7f9f78ff9640 1 -- 172.21.15.98:0/2213993850 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f74004130 con 0x7f9f7c15c470 2024-09-10T20:58:56.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.781+0000 7f9f78ff9640 1 -- 172.21.15.98:0/2213993850 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f74004560 con 0x7f9f7c15c470 2024-09-10T20:58:56.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.781+0000 7f9f82790640 1 -- 172.21.15.98:0/2213993850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7c1f2f80 con 0x7f9f7c15c470 2024-09-10T20:58:56.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.781+0000 7f9f82790640 1 -- 172.21.15.98:0/2213993850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f7c0ab280 con 0x7f9f7c15c470 2024-09-10T20:58:56.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.782+0000 7f9f82790640 1 -- 172.21.15.98:0/2213993850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f7c070080 con 0x7f9f7c15c470 2024-09-10T20:58:56.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.783+0000 7f9f78ff9640 1 -- 172.21.15.98:0/2213993850 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9f7403c420 con 0x7f9f7c15c470 2024-09-10T20:58:56.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.783+0000 7f9f78ff9640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9f68077f70 0x7f9f6807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:56.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.784+0000 7f9f78ff9640 1 -- 172.21.15.98:0/2213993850 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9f740bbac0 con 0x7f9f7c15c470 2024-09-10T20:58:56.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.784+0000 7f9f7bfff640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9f68077f70 0x7f9f6807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:56.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.784+0000 7f9f7bfff640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9f68077f70 0x7f9f6807a430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009b40 tx=0x7f9f6c005e50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:56.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.788+0000 7f9f78ff9640 1 -- 172.21.15.98:0/2213993850 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9f74084310 con 0x7f9f7c15c470 2024-09-10T20:58:56.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.972+0000 7f9f82790640 1 -- 172.21.15.98:0/2213993850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9f7c141170 con 0x7f9f7c15c470 2024-09-10T20:58:56.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.973+0000 7f9f78ff9640 1 -- 172.21.15.98:0/2213993850 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f9f74083c10 con 0x7f9f7c15c470 2024-09-10T20:58:56.977 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-10T20:58:56.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.975+0000 7f9f82790640 1 -- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9f68077f70 msgr2=0x7f9f6807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:56.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.975+0000 7f9f82790640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9f68077f70 0x7f9f6807a430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009b40 tx=0x7f9f6c005e50 comp rx=0 tx=0).stop 2024-09-10T20:58:56.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.975+0000 7f9f82790640 1 -- 172.21.15.98:0/2213993850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7c15c470 msgr2=0x7f9f7c0aba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:56.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.975+0000 7f9f82790640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7c15c470 0x7f9f7c0aba80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9f74009ca0 tx=0x7f9f74003970 comp rx=0 tx=0).stop 2024-09-10T20:58:56.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.975+0000 7f9f82790640 1 -- 172.21.15.98:0/2213993850 shutdown_connections 2024-09-10T20:58:56.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.975+0000 7f9f82790640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9f68077f70 0x7f9f6807a430 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:56.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.975+0000 7f9f82790640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f7c15c470 0x7f9f7c0aba80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:56.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.975+0000 7f9f82790640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c073780 0x7f9f7c0aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:56.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.975+0000 7f9f82790640 1 --2- 172.21.15.98:0/2213993850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c06f140 0x7f9f7c140c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:56.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.975+0000 7f9f82790640 1 -- 172.21.15.98:0/2213993850 >> 172.21.15.98:0/2213993850 conn(0x7f9f7c0b1760 msgr2=0x7f9f7c0ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:56.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.975+0000 7f9f82790640 1 -- 172.21.15.98:0/2213993850 shutdown_connections 2024-09-10T20:58:56.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:56.975+0000 7f9f82790640 1 -- 172.21.15.98:0/2213993850 wait complete. 2024-09-10T20:58:56.990 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:58:56.990 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:58:57.011 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T20:58:57.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.148+0000 7fddcb0fe640 1 -- 172.21.15.98:0/2624581693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddc40d2a20 msgr2=0x7fddc40cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:57.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.148+0000 7fddcb0fe640 1 --2- 172.21.15.98:0/2624581693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddc40d2a20 0x7fddc40cea40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fddbc005fd0 tx=0x7fddbc02a8b0 comp rx=0 tx=0).stop 2024-09-10T20:58:57.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.148+0000 7fddcb0fe640 1 -- 172.21.15.98:0/2624581693 shutdown_connections 2024-09-10T20:58:57.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.148+0000 7fddcb0fe640 1 --2- 172.21.15.98:0/2624581693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddc40d2a20 0x7fddc40cea40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:57.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.148+0000 7fddcb0fe640 1 --2- 172.21.15.98:0/2624581693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddc40d4af0 0x7fddc40d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:57.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.148+0000 7fddcb0fe640 1 -- 172.21.15.98:0/2624581693 >> 172.21.15.98:0/2624581693 conn(0x7fddc40d3f70 msgr2=0x7fddc40d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:57.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.148+0000 7fddcb0fe640 1 -- 172.21.15.98:0/2624581693 shutdown_connections 2024-09-10T20:58:57.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.148+0000 7fddcb0fe640 1 -- 172.21.15.98:0/2624581693 wait complete. 2024-09-10T20:58:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.149+0000 7fddcb0fe640 1 Processor -- start 2024-09-10T20:58:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.150+0000 7fddcb0fe640 1 -- start start 2024-09-10T20:58:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.150+0000 7fddcb0fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddc40d2a20 0x7fddc415a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.150+0000 7fddcb0fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddc40d4af0 0x7fddc40cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.150+0000 7fddcb0fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddc415e060 0x7fddc40b8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.150+0000 7fddcb0fe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fddc40cf100 con 0x7fddc40d4af0 2024-09-10T20:58:57.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.150+0000 7fddcb0fe640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fddc40cef80 con 0x7fddc40d2a20 2024-09-10T20:58:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.150+0000 7fddcb0fe640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fddc40cf280 con 0x7fddc415e060 2024-09-10T20:58:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddc40d4af0 0x7fddc40cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddc40d4af0 0x7fddc40cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:52008/0 (socket says 172.21.15.98:52008) 2024-09-10T20:58:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc3fff640 1 -- 172.21.15.98:0/499974688 learned_addr learned my addr 172.21.15.98:0/499974688 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:58:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc969b640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddc415e060 0x7fddc40b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc8e9a640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddc40d2a20 0x7fddc415a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc3fff640 1 -- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddc415e060 msgr2=0x7fddc40b8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc3fff640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddc415e060 0x7fddc40b8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc3fff640 1 -- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddc40d2a20 msgr2=0x7fddc415a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc3fff640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddc40d2a20 0x7fddc415a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc3fff640 1 -- 172.21.15.98:0/499974688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc41f4c80 con 0x7fddc40d4af0 2024-09-10T20:58:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc969b640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddc415e060 0x7fddc40b8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:58:57.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc3fff640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddc40d4af0 0x7fddc40cdc70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fddb000c890 tx=0x7fddb000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:57.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc17fa640 1 -- 172.21.15.98:0/499974688 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0004220 con 0x7fddc40d4af0 2024-09-10T20:58:57.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc17fa640 1 -- 172.21.15.98:0/499974688 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddb0007950 con 0x7fddc40d4af0 2024-09-10T20:58:57.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddc17fa640 1 -- 172.21.15.98:0/499974688 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0007d80 con 0x7fddc40d4af0 2024-09-10T20:58:57.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddcb0fe640 1 -- 172.21.15.98:0/499974688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc41f4f60 con 0x7fddc40d4af0 2024-09-10T20:58:57.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.151+0000 7fddcb0fe640 1 -- 172.21.15.98:0/499974688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc41f5520 con 0x7fddc40d4af0 2024-09-10T20:58:57.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.152+0000 7fddcb0fe640 1 -- 172.21.15.98:0/499974688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc40cf730 con 0x7fddc40d4af0 2024-09-10T20:58:57.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.153+0000 7fddc17fa640 1 -- 172.21.15.98:0/499974688 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fddb0005020 con 0x7fddc40d4af0 2024-09-10T20:58:57.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.153+0000 7fddc17fa640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdda8077f70 0x7fdda807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:57.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.153+0000 7fddc17fa640 1 -- 172.21.15.98:0/499974688 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fddb009a4f0 con 0x7fddc40d4af0 2024-09-10T20:58:57.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.153+0000 7fddc8e9a640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdda8077f70 0x7fdda807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:57.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.154+0000 7fddc8e9a640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdda8077f70 0x7fdda807a430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000f60 tx=0x7fddb8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:57.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.157+0000 7fddc17fa640 1 -- 172.21.15.98:0/499974688 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fddb0014030 con 0x7fddc40d4af0 2024-09-10T20:58:57.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.346+0000 7fddcb0fe640 1 -- 172.21.15.98:0/499974688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fddc415add0 con 0x7fddc40d4af0 2024-09-10T20:58:57.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.346+0000 7fddc17fa640 1 -- 172.21.15.98:0/499974688 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fddb0062d40 con 0x7fddc40d4af0 2024-09-10T20:58:57.350 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-10T20:58:57.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.348+0000 7fddcb0fe640 1 -- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdda8077f70 msgr2=0x7fdda807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:57.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.348+0000 7fddcb0fe640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdda8077f70 0x7fdda807a430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000f60 tx=0x7fddb8009040 comp rx=0 tx=0).stop 2024-09-10T20:58:57.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.348+0000 7fddcb0fe640 1 -- 172.21.15.98:0/499974688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddc40d4af0 msgr2=0x7fddc40cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:57.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.348+0000 7fddcb0fe640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddc40d4af0 0x7fddc40cdc70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fddb000c890 tx=0x7fddb000cd50 comp rx=0 tx=0).stop 2024-09-10T20:58:57.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.349+0000 7fddcb0fe640 1 -- 172.21.15.98:0/499974688 shutdown_connections 2024-09-10T20:58:57.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.349+0000 7fddcb0fe640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdda8077f70 0x7fdda807a430 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:57.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.349+0000 7fddcb0fe640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddc415e060 0x7fddc40b8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:57.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.349+0000 7fddcb0fe640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fddc40d4af0 0x7fddc40cdc70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:57.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.349+0000 7fddcb0fe640 1 --2- 172.21.15.98:0/499974688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddc40d2a20 0x7fddc415a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:57.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.349+0000 7fddcb0fe640 1 -- 172.21.15.98:0/499974688 >> 172.21.15.98:0/499974688 conn(0x7fddc40d3f70 msgr2=0x7fddc40b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:57.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.349+0000 7fddcb0fe640 1 -- 172.21.15.98:0/499974688 shutdown_connections 2024-09-10T20:58:57.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:58:57.349+0000 7fddcb0fe640 1 -- 172.21.15.98:0/499974688 wait complete. 2024-09-10T20:58:57.363 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2024-09-10T20:58:57.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:58:57.364 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-10T20:58:57.386 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi098.stderr:ceph-fuse[43543]: fuse finished with error 0 and tester_r 0 2024-09-10T20:58:57.388 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:59:03.490 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T20:59:03.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:03.490 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-10T20:59:03.521 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:59:03.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.668+0000 7f0db998a640 1 -- 172.21.15.98:0/3155750838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db40d2300 msgr2=0x7f0db40d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:03.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.668+0000 7f0db998a640 1 --2- 172.21.15.98:0/3155750838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db40d2300 0x7f0db40d2730 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c005fd0 tx=0x7f0d9c02a540 comp rx=0 tx=0).stop 2024-09-10T20:59:03.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.668+0000 7f0db998a640 1 -- 172.21.15.98:0/3155750838 shutdown_connections 2024-09-10T20:59:03.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.668+0000 7f0db998a640 1 --2- 172.21.15.98:0/3155750838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db40d2300 0x7f0db40d2730 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.668+0000 7f0db998a640 1 -- 172.21.15.98:0/3155750838 >> 172.21.15.98:0/3155750838 conn(0x7f0db40d3db0 msgr2=0x7f0db40d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:03.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.668+0000 7f0db998a640 1 -- 172.21.15.98:0/3155750838 shutdown_connections 2024-09-10T20:59:03.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.668+0000 7f0db998a640 1 -- 172.21.15.98:0/3155750838 wait complete. 2024-09-10T20:59:03.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.669+0000 7f0db998a640 1 Processor -- start 2024-09-10T20:59:03.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.670+0000 7f0db998a640 1 -- start start 2024-09-10T20:59:03.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.670+0000 7f0db998a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db40b2410 0x7f0db40a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:03.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.670+0000 7f0db998a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0db40d2300 0x7f0db40b8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:03.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.670+0000 7f0db998a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0db415dea0 0x7f0db4156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:03.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.670+0000 7f0db998a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0db40b6c00 con 0x7f0db40b2410 2024-09-10T20:59:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.670+0000 7f0db998a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0db40b6d80 con 0x7f0db40d2300 2024-09-10T20:59:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0db998a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0db40b2fb0 con 0x7f0db415dea0 2024-09-10T20:59:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0db2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db40b2410 0x7f0db40a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0db2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db40b2410 0x7f0db40a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58294/0 (socket says 172.21.15.98:58294) 2024-09-10T20:59:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0db2ffd640 1 -- 172.21.15.98:0/3584995311 learned_addr learned my addr 172.21.15.98:0/3584995311 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0db27fc640 1 --2- 172.21.15.98:0/3584995311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0db40d2300 0x7f0db40b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0db37fe640 1 --2- 172.21.15.98:0/3584995311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0db415dea0 0x7f0db4156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0db2ffd640 1 -- 172.21.15.98:0/3584995311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0db415dea0 msgr2=0x7f0db4156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:03.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0db2ffd640 1 --2- 172.21.15.98:0/3584995311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0db415dea0 0x7f0db4156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0db2ffd640 1 -- 172.21.15.98:0/3584995311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0db40d2300 msgr2=0x7f0db40b8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:03.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0db2ffd640 1 --2- 172.21.15.98:0/3584995311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0db40d2300 0x7f0db40b8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0db2ffd640 1 -- 172.21.15.98:0/3584995311 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db40a0cb0 con 0x7f0db40b2410 2024-09-10T20:59:03.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0db2ffd640 1 --2- 172.21.15.98:0/3584995311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db40b2410 0x7f0db40a0770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0da400c730 tx=0x7f0da400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:03.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0dabfff640 1 -- 172.21.15.98:0/3584995311 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4003b70 con 0x7f0db40b2410 2024-09-10T20:59:03.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0dabfff640 1 -- 172.21.15.98:0/3584995311 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0da4003d30 con 0x7f0db40b2410 2024-09-10T20:59:03.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0db998a640 1 -- 172.21.15.98:0/3584995311 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db409dab0 con 0x7f0db40b2410 2024-09-10T20:59:03.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0dabfff640 1 -- 172.21.15.98:0/3584995311 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da400aab0 con 0x7f0db40b2410 2024-09-10T20:59:03.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.671+0000 7f0db998a640 1 -- 172.21.15.98:0/3584995311 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db40a1210 con 0x7f0db40b2410 2024-09-10T20:59:03.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.672+0000 7f0db998a640 1 -- 172.21.15.98:0/3584995311 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db40d2730 con 0x7f0db40b2410 2024-09-10T20:59:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.673+0000 7f0dabfff640 1 -- 172.21.15.98:0/3584995311 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0da400ac50 con 0x7f0db40b2410 2024-09-10T20:59:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.673+0000 7f0dabfff640 1 --2- 172.21.15.98:0/3584995311 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0d94077f70 0x7f0d9407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.673+0000 7f0dabfff640 1 -- 172.21.15.98:0/3584995311 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0da4099f20 con 0x7f0db40b2410 2024-09-10T20:59:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.673+0000 7f0db27fc640 1 --2- 172.21.15.98:0/3584995311 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0d94077f70 0x7f0d9407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:03.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.674+0000 7f0db27fc640 1 --2- 172.21.15.98:0/3584995311 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0d94077f70 0x7f0d9407a430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c033000 tx=0x7f0d9c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:03.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.677+0000 7f0dabfff640 1 -- 172.21.15.98:0/3584995311 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0db40a0cb0 con 0x7f0db40b2410 2024-09-10T20:59:03.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.851+0000 7f0db998a640 1 -- 172.21.15.98:0/3584995311 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0db40d02e0 con 0x7f0db40b2410 2024-09-10T20:59:03.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.852+0000 7f0dabfff640 1 -- 172.21.15.98:0/3584995311 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18610 (secure 0 0 0) 0x7f0da40626e0 con 0x7f0db40b2410 2024-09-10T20:59:03.856 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:03.857 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":25,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:58:51.602367+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:58:44.390290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T20:58:45.564099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T20:58:49.250508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:59:03.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.854+0000 7f0db998a640 1 -- 172.21.15.98:0/3584995311 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0d94077f70 msgr2=0x7f0d9407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:03.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.854+0000 7f0db998a640 1 --2- 172.21.15.98:0/3584995311 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0d94077f70 0x7f0d9407a430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c033000 tx=0x7f0d9c004730 comp rx=0 tx=0).stop 2024-09-10T20:59:03.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.854+0000 7f0db998a640 1 -- 172.21.15.98:0/3584995311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db40b2410 msgr2=0x7f0db40a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:03.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.854+0000 7f0db998a640 1 --2- 172.21.15.98:0/3584995311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db40b2410 0x7f0db40a0770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0da400c730 tx=0x7f0da400cbf0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.854+0000 7f0db998a640 1 -- 172.21.15.98:0/3584995311 shutdown_connections 2024-09-10T20:59:03.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.854+0000 7f0db998a640 1 --2- 172.21.15.98:0/3584995311 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0d94077f70 0x7f0d9407a430 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.854+0000 7f0db998a640 1 --2- 172.21.15.98:0/3584995311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0db415dea0 0x7f0db4156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.854+0000 7f0db998a640 1 --2- 172.21.15.98:0/3584995311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0db40d2300 0x7f0db40b8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.854+0000 7f0db998a640 1 --2- 172.21.15.98:0/3584995311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0db40b2410 0x7f0db40a0770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.854+0000 7f0db998a640 1 -- 172.21.15.98:0/3584995311 >> 172.21.15.98:0/3584995311 conn(0x7f0db40d3db0 msgr2=0x7f0db406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:03.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.854+0000 7f0db998a640 1 -- 172.21.15.98:0/3584995311 shutdown_connections 2024-09-10T20:59:03.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.854+0000 7f0db998a640 1 -- 172.21.15.98:0/3584995311 wait complete. 2024-09-10T20:59:03.870 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:59:03.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.967+0000 7f1c3833a640 1 -- 172.21.15.98:0/4270098767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c3006dcc0 msgr2=0x7f1c300b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:03.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.967+0000 7f1c3833a640 1 --2- 172.21.15.98:0/4270098767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c3006dcc0 0x7f1c300b1740 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1c30072cf0 tx=0x7f1c2c02a440 comp rx=0 tx=0).stop 2024-09-10T20:59:03.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.967+0000 7f1c3833a640 1 -- 172.21.15.98:0/4270098767 shutdown_connections 2024-09-10T20:59:03.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.967+0000 7f1c3833a640 1 --2- 172.21.15.98:0/4270098767 >> v2:172.21.15.190:3300/0 conn(0x7f1c3006cb70 0x7f1c300cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.967+0000 7f1c3833a640 1 --2- 172.21.15.98:0/4270098767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c3006dcc0 0x7f1c300b1740 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.967+0000 7f1c3833a640 1 -- 172.21.15.98:0/4270098767 >> 172.21.15.98:0/4270098767 conn(0x7f1c30074150 msgr2=0x7f1c30074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:03.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.967+0000 7f1c3833a640 1 -- 172.21.15.98:0/4270098767 shutdown_connections 2024-09-10T20:59:03.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.967+0000 7f1c3833a640 1 -- 172.21.15.98:0/4270098767 wait complete. 2024-09-10T20:59:03.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.968+0000 7f1c3833a640 1 Processor -- start 2024-09-10T20:59:03.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.968+0000 7f1c3833a640 1 -- start start 2024-09-10T20:59:03.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c3833a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c3006cb70 0x7f1c3015e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:03.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c3833a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c3006dcc0 0x7f1c30153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:03.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c3833a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c3006e0a0 0x7f1c30099520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:03.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c3833a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1c300d0320 con 0x7f1c3006cb70 2024-09-10T20:59:03.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c3833a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1c300cb5c0 con 0x7f1c3006e0a0 2024-09-10T20:59:03.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c3833a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1c300d04a0 con 0x7f1c3006dcc0 2024-09-10T20:59:03.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c360d6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c3006cb70 0x7f1c3015e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:03.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c360d6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c3006cb70 0x7f1c3015e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58304/0 (socket says 172.21.15.98:58304) 2024-09-10T20:59:03.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c360d6640 1 -- 172.21.15.98:0/2860427979 learned_addr learned my addr 172.21.15.98:0/2860427979 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:03.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c358d5640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c3006dcc0 0x7f1c30153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:03.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c368d7640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c3006e0a0 0x7f1c30099520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:03.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c360d6640 1 -- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c3006dcc0 msgr2=0x7f1c30153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:03.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c360d6640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c3006dcc0 0x7f1c30153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c360d6640 1 -- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c3006e0a0 msgr2=0x7f1c30099520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:03.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c360d6640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c3006e0a0 0x7f1c30099520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c360d6640 1 -- 172.21.15.98:0/2860427979 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3015bb20 con 0x7f1c3006cb70 2024-09-10T20:59:03.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c368d7640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c3006e0a0 0x7f1c30099520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:03.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c360d6640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c3006cb70 0x7f1c3015e090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c02aa30 tx=0x7f1c2c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:03.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c26ffd640 1 -- 172.21.15.98:0/2860427979 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c03f070 con 0x7f1c3006cb70 2024-09-10T20:59:03.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c26ffd640 1 -- 172.21.15.98:0/2860427979 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c2c004890 con 0x7f1c3006cb70 2024-09-10T20:59:03.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c3833a640 1 -- 172.21.15.98:0/2860427979 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3015c940 con 0x7f1c3006cb70 2024-09-10T20:59:03.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c26ffd640 1 -- 172.21.15.98:0/2860427979 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c035030 con 0x7f1c3006cb70 2024-09-10T20:59:03.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.969+0000 7f1c3833a640 1 -- 172.21.15.98:0/2860427979 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c300cf2a0 con 0x7f1c3006cb70 2024-09-10T20:59:03.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.970+0000 7f1c3833a640 1 -- 172.21.15.98:0/2860427979 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c300cc5f0 con 0x7f1c3006cb70 2024-09-10T20:59:03.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.971+0000 7f1c26ffd640 1 -- 172.21.15.98:0/2860427979 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1c2c004ce0 con 0x7f1c3006cb70 2024-09-10T20:59:03.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.971+0000 7f1c26ffd640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1c14077f70 0x7f1c1407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:03.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.971+0000 7f1c26ffd640 1 -- 172.21.15.98:0/2860427979 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1c2c0bb310 con 0x7f1c3006cb70 2024-09-10T20:59:03.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.971+0000 7f1c358d5640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1c14077f70 0x7f1c1407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:03.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.972+0000 7f1c358d5640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1c14077f70 0x7f1c1407a430 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1c20006fd0 tx=0x7f1c20008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:03.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:03.975+0000 7f1c26ffd640 1 -- 172.21.15.98:0/2860427979 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1c300cc5f0 con 0x7f1c3006cb70 2024-09-10T20:59:04.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.150+0000 7f1c3833a640 1 -- 172.21.15.98:0/2860427979 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c3006f030 con 0x7f1c3006cb70 2024-09-10T20:59:04.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.151+0000 7f1c26ffd640 1 -- 172.21.15.98:0/2860427979 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18610 (secure 0 0 0) 0x7f1c3006f030 con 0x7f1c3006cb70 2024-09-10T20:59:04.156 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:04.156 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":25,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:58:51.602367+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:58:44.390290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T20:58:45.564099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T20:58:49.250508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:59:04.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.154+0000 7f1c3833a640 1 -- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1c14077f70 msgr2=0x7f1c1407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:04.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.154+0000 7f1c3833a640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1c14077f70 0x7f1c1407a430 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1c20006fd0 tx=0x7f1c20008040 comp rx=0 tx=0).stop 2024-09-10T20:59:04.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.154+0000 7f1c3833a640 1 -- 172.21.15.98:0/2860427979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c3006cb70 msgr2=0x7f1c3015e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:04.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.154+0000 7f1c3833a640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c3006cb70 0x7f1c3015e090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c02aa30 tx=0x7f1c2c0039e0 comp rx=0 tx=0).stop 2024-09-10T20:59:04.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.154+0000 7f1c3833a640 1 -- 172.21.15.98:0/2860427979 shutdown_connections 2024-09-10T20:59:04.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.154+0000 7f1c3833a640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1c14077f70 0x7f1c1407a430 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:04.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.154+0000 7f1c3833a640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c3006e0a0 0x7f1c30099520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:04.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.154+0000 7f1c3833a640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1c3006dcc0 0x7f1c30153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:04.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.154+0000 7f1c3833a640 1 --2- 172.21.15.98:0/2860427979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c3006cb70 0x7f1c3015e090 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:04.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.154+0000 7f1c3833a640 1 -- 172.21.15.98:0/2860427979 >> 172.21.15.98:0/2860427979 conn(0x7f1c30074150 msgr2=0x7f1c300ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:04.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.154+0000 7f1c3833a640 1 -- 172.21.15.98:0/2860427979 shutdown_connections 2024-09-10T20:59:04.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.154+0000 7f1c3833a640 1 -- 172.21.15.98:0/2860427979 wait complete. 2024-09-10T20:59:04.169 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T20:59:04.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.269+0000 7fbb816ef640 1 -- 172.21.15.98:0/2700530349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb68004800 msgr2=0x7fbb68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:04.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.269+0000 7fbb816ef640 1 --2- 172.21.15.98:0/2700530349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb68004800 0x7fbb68004c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009b00 tx=0x7fbb68005830 comp rx=0 tx=0).stop 2024-09-10T20:59:04.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.269+0000 7fbb816ef640 1 -- 172.21.15.98:0/2700530349 shutdown_connections 2024-09-10T20:59:04.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.269+0000 7fbb816ef640 1 --2- 172.21.15.98:0/2700530349 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb68004800 0x7fbb68004c80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:04.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.269+0000 7fbb816ef640 1 -- 172.21.15.98:0/2700530349 >> 172.21.15.98:0/2700530349 conn(0x7fbb7c0d3f70 msgr2=0x7fbb7c0d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:04.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.269+0000 7fbb816ef640 1 -- 172.21.15.98:0/2700530349 shutdown_connections 2024-09-10T20:59:04.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.269+0000 7fbb816ef640 1 -- 172.21.15.98:0/2700530349 wait complete. 2024-09-10T20:59:04.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.270+0000 7fbb816ef640 1 Processor -- start 2024-09-10T20:59:04.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.270+0000 7fbb816ef640 1 -- start start 2024-09-10T20:59:04.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb816ef640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb68004800 0x7fbb7c0c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:04.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb816ef640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb7c0ca670 0x7fbb7c0c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:04.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb816ef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb7c1f8390 0x7fbb7c1f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:04.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb816ef640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c0d24e0 con 0x7fbb68004800 2024-09-10T20:59:04.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb816ef640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c0d2f80 con 0x7fbb7c1f8390 2024-09-10T20:59:04.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb816ef640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbb7c0d2660 con 0x7fbb7c0ca670 2024-09-10T20:59:04.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb7affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb68004800 0x7fbb7c0c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:04.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb7affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb68004800 0x7fbb7c0c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58336/0 (socket says 172.21.15.98:58336) 2024-09-10T20:59:04.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb7affd640 1 -- 172.21.15.98:0/2023788343 learned_addr learned my addr 172.21.15.98:0/2023788343 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:04.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb7b7fe640 1 --2- 172.21.15.98:0/2023788343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb7c1f8390 0x7fbb7c1f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:04.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb7a7fc640 1 --2- 172.21.15.98:0/2023788343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb7c0ca670 0x7fbb7c0c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:04.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb7affd640 1 -- 172.21.15.98:0/2023788343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb7c0ca670 msgr2=0x7fbb7c0c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:04.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb7affd640 1 --2- 172.21.15.98:0/2023788343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb7c0ca670 0x7fbb7c0c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:04.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb7affd640 1 -- 172.21.15.98:0/2023788343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb7c1f8390 msgr2=0x7fbb7c1f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:04.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb7affd640 1 --2- 172.21.15.98:0/2023788343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb7c1f8390 0x7fbb7c1f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:04.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.271+0000 7fbb7affd640 1 -- 172.21.15.98:0/2023788343 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c1f5830 con 0x7fbb68004800 2024-09-10T20:59:04.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.272+0000 7fbb7affd640 1 --2- 172.21.15.98:0/2023788343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb68004800 0x7fbb7c0c7700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0013d0 tx=0x7fbb6c006810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:04.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.272+0000 7fbb737fe640 1 -- 172.21.15.98:0/2023788343 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c00fae0 con 0x7fbb68004800 2024-09-10T20:59:04.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.272+0000 7fbb737fe640 1 -- 172.21.15.98:0/2023788343 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb6c016020 con 0x7fbb68004800 2024-09-10T20:59:04.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.272+0000 7fbb816ef640 1 -- 172.21.15.98:0/2023788343 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7c0c98d0 con 0x7fbb68004800 2024-09-10T20:59:04.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.272+0000 7fbb737fe640 1 -- 172.21.15.98:0/2023788343 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c011430 con 0x7fbb68004800 2024-09-10T20:59:04.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.272+0000 7fbb816ef640 1 -- 172.21.15.98:0/2023788343 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb7c0c9e90 con 0x7fbb68004800 2024-09-10T20:59:04.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.273+0000 7fbb816ef640 1 -- 172.21.15.98:0/2023788343 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb7c0b6e50 con 0x7fbb68004800 2024-09-10T20:59:04.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.273+0000 7fbb737fe640 1 -- 172.21.15.98:0/2023788343 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbb6c0115d0 con 0x7fbb68004800 2024-09-10T20:59:04.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.274+0000 7fbb737fe640 1 --2- 172.21.15.98:0/2023788343 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbb5c077f70 0x7fbb5c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:04.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.274+0000 7fbb737fe640 1 -- 172.21.15.98:0/2023788343 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbb6c09ada0 con 0x7fbb68004800 2024-09-10T20:59:04.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.274+0000 7fbb7a7fc640 1 --2- 172.21.15.98:0/2023788343 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbb5c077f70 0x7fbb5c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:04.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.274+0000 7fbb7a7fc640 1 --2- 172.21.15.98:0/2023788343 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbb5c077f70 0x7fbb5c07a430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbb64034000 tx=0x7fbb640046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:04.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.278+0000 7fbb737fe640 1 -- 172.21.15.98:0/2023788343 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbb6c020020 con 0x7fbb68004800 2024-09-10T20:59:04.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:04.493+0000 7fbb816ef640 1 -- 172.21.15.98:0/2023788343 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbb7c0b9970 con 0x7fbb68004800 2024-09-10T20:59:05.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.293+0000 7fbb737fe640 1 -- 172.21.15.98:0/2023788343 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fbb6c062f10 con 0x7fbb68004800 2024-09-10T20:59:05.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.295+0000 7fbb816ef640 1 -- 172.21.15.98:0/2023788343 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbb5c077f70 msgr2=0x7fbb5c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:05.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.295+0000 7fbb816ef640 1 --2- 172.21.15.98:0/2023788343 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbb5c077f70 0x7fbb5c07a430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbb64034000 tx=0x7fbb640046c0 comp rx=0 tx=0).stop 2024-09-10T20:59:05.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.295+0000 7fbb816ef640 1 -- 172.21.15.98:0/2023788343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb68004800 msgr2=0x7fbb7c0c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:05.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.295+0000 7fbb816ef640 1 --2- 172.21.15.98:0/2023788343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb68004800 0x7fbb7c0c7700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0013d0 tx=0x7fbb6c006810 comp rx=0 tx=0).stop 2024-09-10T20:59:05.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.296+0000 7fbb816ef640 1 -- 172.21.15.98:0/2023788343 shutdown_connections 2024-09-10T20:59:05.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.296+0000 7fbb816ef640 1 --2- 172.21.15.98:0/2023788343 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbb5c077f70 0x7fbb5c07a430 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:05.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.296+0000 7fbb816ef640 1 --2- 172.21.15.98:0/2023788343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb7c1f8390 0x7fbb7c1f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:05.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.296+0000 7fbb816ef640 1 --2- 172.21.15.98:0/2023788343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb7c0ca670 0x7fbb7c0c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:05.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.296+0000 7fbb816ef640 1 --2- 172.21.15.98:0/2023788343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb68004800 0x7fbb7c0c7700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:05.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.296+0000 7fbb816ef640 1 -- 172.21.15.98:0/2023788343 >> 172.21.15.98:0/2023788343 conn(0x7fbb7c0d3f70 msgr2=0x7fbb7c161a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:05.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.296+0000 7fbb816ef640 1 -- 172.21.15.98:0/2023788343 shutdown_connections 2024-09-10T20:59:05.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.296+0000 7fbb816ef640 1 -- 172.21.15.98:0/2023788343 wait complete. 2024-09-10T20:59:05.312 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-10T20:59:05.312 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T20:59:05.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.416+0000 7f099afc6640 1 -- 172.21.15.98:0/901615935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0994144780 msgr2=0x7f09940b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:05.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.416+0000 7f099afc6640 1 --2- 172.21.15.98:0/901615935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0994144780 0x7f09940b3da0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f099415e6b0 tx=0x7f098402a820 comp rx=0 tx=0).stop 2024-09-10T20:59:05.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.416+0000 7f099afc6640 1 -- 172.21.15.98:0/901615935 shutdown_connections 2024-09-10T20:59:05.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.416+0000 7f099afc6640 1 --2- 172.21.15.98:0/901615935 >> v2:172.21.15.190:3300/0 conn(0x7f0994072680 0x7f09940ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:05.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.416+0000 7f099afc6640 1 --2- 172.21.15.98:0/901615935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0994144780 0x7f09940b3da0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:05.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.416+0000 7f099afc6640 1 -- 172.21.15.98:0/901615935 >> 172.21.15.98:0/901615935 conn(0x7f09940b1760 msgr2=0x7f09940b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:05.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.416+0000 7f099afc6640 1 -- 172.21.15.98:0/901615935 shutdown_connections 2024-09-10T20:59:05.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.416+0000 7f099afc6640 1 -- 172.21.15.98:0/901615935 wait complete. 2024-09-10T20:59:05.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.417+0000 7f099afc6640 1 Processor -- start 2024-09-10T20:59:05.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.417+0000 7f099afc6640 1 -- start start 2024-09-10T20:59:05.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.418+0000 7f099afc6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0994072680 0x7f09941f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:05.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.418+0000 7f099afc6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0994072ad0 0x7f09941f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:05.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.418+0000 7f099afc6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0994144780 0x7f09940ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:05.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.418+0000 7f099afc6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f09940cf250 con 0x7f0994144780 2024-09-10T20:59:05.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.418+0000 7f099afc6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f099415c1c0 con 0x7f0994072680 2024-09-10T20:59:05.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.418+0000 7f099afc6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f09940cf3d0 con 0x7f0994072ad0 2024-09-10T20:59:05.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.418+0000 7f0999563640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0994144780 0x7f09940ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:05.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.418+0000 7f0999563640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0994144780 0x7f09940ac710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58366/0 (socket says 172.21.15.98:58366) 2024-09-10T20:59:05.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.418+0000 7f0999563640 1 -- 172.21.15.98:0/3088078503 learned_addr learned my addr 172.21.15.98:0/3088078503 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:05.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.418+0000 7f0998d62640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0994072680 0x7f09941f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:05.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.419+0000 7f0993fff640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0994072ad0 0x7f09941f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:05.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.419+0000 7f0999563640 1 -- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0994072ad0 msgr2=0x7f09941f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:05.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.419+0000 7f0999563640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0994072ad0 0x7f09941f33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:05.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.419+0000 7f0999563640 1 -- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0994072680 msgr2=0x7f09941f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:05.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.419+0000 7f0999563640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0994072680 0x7f09941f60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:05.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.419+0000 7f0999563640 1 -- 172.21.15.98:0/3088078503 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09940af830 con 0x7f0994144780 2024-09-10T20:59:05.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.419+0000 7f0993fff640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0994072ad0 0x7f09941f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:05.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.419+0000 7f0999563640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0994144780 0x7f09940ac710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c3d0 tx=0x7f098c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:05.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.419+0000 7f09917fa640 1 -- 172.21.15.98:0/3088078503 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c017070 con 0x7f0994144780 2024-09-10T20:59:05.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.419+0000 7f09917fa640 1 -- 172.21.15.98:0/3088078503 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f098c0035f0 con 0x7f0994144780 2024-09-10T20:59:05.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.419+0000 7f099afc6640 1 -- 172.21.15.98:0/3088078503 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09940afb10 con 0x7f0994144780 2024-09-10T20:59:05.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.419+0000 7f09917fa640 1 -- 172.21.15.98:0/3088078503 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c003a40 con 0x7f0994144780 2024-09-10T20:59:05.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.419+0000 7f099afc6640 1 -- 172.21.15.98:0/3088078503 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09941f65e0 con 0x7f0994144780 2024-09-10T20:59:05.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.420+0000 7f099afc6640 1 -- 172.21.15.98:0/3088078503 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f099415cf20 con 0x7f0994144780 2024-09-10T20:59:05.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.420+0000 7f09917fa640 1 -- 172.21.15.98:0/3088078503 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f098c003be0 con 0x7f0994144780 2024-09-10T20:59:05.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.421+0000 7f09917fa640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0974077f70 0x7f097407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:05.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.421+0000 7f09917fa640 1 -- 172.21.15.98:0/3088078503 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f098c099600 con 0x7f0994144780 2024-09-10T20:59:05.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.421+0000 7f0998d62640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0974077f70 0x7f097407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:05.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.422+0000 7f0998d62640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0974077f70 0x7f097407a430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0984034000 tx=0x7f0984004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:05.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.425+0000 7f09917fa640 1 -- 172.21.15.98:0/3088078503 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f098c061e50 con 0x7f0994144780 2024-09-10T20:59:05.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:05.634+0000 7f099afc6640 1 -- 172.21.15.98:0/3088078503 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f09940b4d40 con 0x7f0994144780 2024-09-10T20:59:06.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.294+0000 7f09917fa640 1 -- 172.21.15.98:0/3088078503 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f098c061750 con 0x7f0994144780 2024-09-10T20:59:06.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.297+0000 7f099afc6640 1 -- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0974077f70 msgr2=0x7f097407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:06.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.297+0000 7f099afc6640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0974077f70 0x7f097407a430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0984034000 tx=0x7f0984004ab0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.297+0000 7f099afc6640 1 -- 172.21.15.98:0/3088078503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0994144780 msgr2=0x7f09940ac710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:06.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.297+0000 7f099afc6640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0994144780 0x7f09940ac710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c3d0 tx=0x7f098c00c890 comp rx=0 tx=0).stop 2024-09-10T20:59:06.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.297+0000 7f099afc6640 1 -- 172.21.15.98:0/3088078503 shutdown_connections 2024-09-10T20:59:06.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.297+0000 7f099afc6640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0974077f70 0x7f097407a430 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.297+0000 7f099afc6640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0994144780 0x7f09940ac710 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.297+0000 7f099afc6640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0994072ad0 0x7f09941f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.297+0000 7f099afc6640 1 --2- 172.21.15.98:0/3088078503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0994072680 0x7f09941f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.297+0000 7f099afc6640 1 -- 172.21.15.98:0/3088078503 >> 172.21.15.98:0/3088078503 conn(0x7f09940b1760 msgr2=0x7f09940cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:06.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.297+0000 7f099afc6640 1 -- 172.21.15.98:0/3088078503 shutdown_connections 2024-09-10T20:59:06.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.297+0000 7f099afc6640 1 -- 172.21.15.98:0/3088078503 wait complete. 2024-09-10T20:59:06.312 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T20:59:06.313 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:59:06.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.412+0000 7f847a1e2640 1 -- 172.21.15.98:0/3948164846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f845c003a50 msgr2=0x7f845c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:06.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.412+0000 7f847a1e2640 1 --2- 172.21.15.98:0/3948164846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f845c003a50 0x7f845c003e70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8464009e20 tx=0x7f845c005830 comp rx=0 tx=0).stop 2024-09-10T20:59:06.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.412+0000 7f8473fff640 1 -- 172.21.15.98:0/3948164846 reap_dead start 2024-09-10T20:59:06.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.412+0000 7f847a1e2640 1 -- 172.21.15.98:0/3948164846 shutdown_connections 2024-09-10T20:59:06.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.412+0000 7f847a1e2640 1 -- 172.21.15.98:0/3948164846 >> 172.21.15.98:0/3948164846 conn(0x7f84740b37c0 msgr2=0x7f84740b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:06.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.412+0000 7f847a1e2640 1 -- 172.21.15.98:0/3948164846 shutdown_connections 2024-09-10T20:59:06.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.412+0000 7f847a1e2640 1 -- 172.21.15.98:0/3948164846 wait complete. 2024-09-10T20:59:06.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.413+0000 7f847a1e2640 1 Processor -- start 2024-09-10T20:59:06.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.414+0000 7f847a1e2640 1 -- start start 2024-09-10T20:59:06.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.414+0000 7f847a1e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845c003a50 0x7f8474155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:06.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.414+0000 7f847a1e2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84740c9e50 0x7f84740c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:06.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.414+0000 7f847a1e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f847406e360 0x7f847406e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:06.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.414+0000 7f847a1e2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f84740b05c0 con 0x7f84740c9e50 2024-09-10T20:59:06.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.414+0000 7f847a1e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f84740b34f0 con 0x7f845c003a50 2024-09-10T20:59:06.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.414+0000 7f847a1e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f84740b0740 con 0x7f847406e360 2024-09-10T20:59:06.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.414+0000 7f8472ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84740c9e50 0x7f84740c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:06.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.414+0000 7f8472ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84740c9e50 0x7f84740c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58384/0 (socket says 172.21.15.98:58384) 2024-09-10T20:59:06.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.414+0000 7f8472ffd640 1 -- 172.21.15.98:0/969048975 learned_addr learned my addr 172.21.15.98:0/969048975 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:06.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.415+0000 7f84737fe640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845c003a50 0x7f8474155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:06.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.415+0000 7f8473fff640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f847406e360 0x7f847406e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:06.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.415+0000 7f8472ffd640 1 -- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f847406e360 msgr2=0x7f847406e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:06.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.415+0000 7f8472ffd640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f847406e360 0x7f847406e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.415+0000 7f8472ffd640 1 -- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845c003a50 msgr2=0x7f8474155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:06.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.415+0000 7f8472ffd640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845c003a50 0x7f8474155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.415+0000 7f8472ffd640 1 -- 172.21.15.98:0/969048975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84740c95b0 con 0x7f84740c9e50 2024-09-10T20:59:06.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.415+0000 7f8473fff640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f847406e360 0x7f847406e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:06.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.415+0000 7f8472ffd640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84740c9e50 0x7f84740c8d60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8468011850 tx=0x7f8468011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:06.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.415+0000 7f845bfff640 1 -- 172.21.15.98:0/969048975 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846800fcf0 con 0x7f84740c9e50 2024-09-10T20:59:06.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.415+0000 7f845bfff640 1 -- 172.21.15.98:0/969048975 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8468005020 con 0x7f84740c9e50 2024-09-10T20:59:06.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.415+0000 7f847a1e2640 1 -- 172.21.15.98:0/969048975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84740c9890 con 0x7f84740c9e50 2024-09-10T20:59:06.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.415+0000 7f845bfff640 1 -- 172.21.15.98:0/969048975 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468005430 con 0x7f84740c9e50 2024-09-10T20:59:06.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.415+0000 7f847a1e2640 1 -- 172.21.15.98:0/969048975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f847406db60 con 0x7f84740c9e50 2024-09-10T20:59:06.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.416+0000 7f847a1e2640 1 -- 172.21.15.98:0/969048975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84740cfa20 con 0x7f84740c9e50 2024-09-10T20:59:06.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.417+0000 7f845bfff640 1 -- 172.21.15.98:0/969048975 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f84680055d0 con 0x7f84740c9e50 2024-09-10T20:59:06.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.417+0000 7f845bfff640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8460077fa0 0x7f846007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:06.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.417+0000 7f845bfff640 1 -- 172.21.15.98:0/969048975 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f846809ab20 con 0x7f84740c9e50 2024-09-10T20:59:06.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.417+0000 7f84737fe640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8460077fa0 0x7f846007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:06.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.418+0000 7f84737fe640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8460077fa0 0x7f846007a460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8464035000 tx=0x7f846402d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:06.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.421+0000 7f845bfff640 1 -- 172.21.15.98:0/969048975 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8468063370 con 0x7f84740c9e50 2024-09-10T20:59:06.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.633+0000 7f847a1e2640 1 -- 172.21.15.98:0/969048975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84740b4590 con 0x7f84740c9e50 2024-09-10T20:59:06.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.634+0000 7f845bfff640 1 -- 172.21.15.98:0/969048975 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7f8468062c70 con 0x7f84740c9e50 2024-09-10T20:59:06.638 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:06.638 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"btime":"2024-09-10T20:59:06:294252+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4277,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/1954552281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1954552281},{"type":"v1","addr":"172.21.15.190:6836","nonce":1954552281}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/8997965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":8997965},{"type":"v1","addr":"172.21.15.190:6837","nonce":8997965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1984550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1984550306},{"type":"v1","addr":"172.21.15.98:6833","nonce":1984550306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:58:46.560479+0000","modified":"2024-09-10T20:59:05.635381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3141703317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3141703317},{"type":"v1","addr":"172.21.15.98:6835","nonce":3141703317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T20:59:06.638 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2024-09-10T20:59:06.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.636+0000 7f847a1e2640 1 -- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8460077fa0 msgr2=0x7f846007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:06.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.636+0000 7f847a1e2640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8460077fa0 0x7f846007a460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8464035000 tx=0x7f846402d040 comp rx=0 tx=0).stop 2024-09-10T20:59:06.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.636+0000 7f847a1e2640 1 -- 172.21.15.98:0/969048975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84740c9e50 msgr2=0x7f84740c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:06.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.636+0000 7f847a1e2640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84740c9e50 0x7f84740c8d60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8468011850 tx=0x7f8468011d10 comp rx=0 tx=0).stop 2024-09-10T20:59:06.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.636+0000 7f847a1e2640 1 -- 172.21.15.98:0/969048975 shutdown_connections 2024-09-10T20:59:06.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.636+0000 7f847a1e2640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8460077fa0 0x7f846007a460 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.636+0000 7f847a1e2640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f847406e360 0x7f847406e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.636+0000 7f847a1e2640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84740c9e50 0x7f84740c8d60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.636+0000 7f847a1e2640 1 --2- 172.21.15.98:0/969048975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f845c003a50 0x7f8474155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.636+0000 7f847a1e2640 1 -- 172.21.15.98:0/969048975 >> 172.21.15.98:0/969048975 conn(0x7f84740b37c0 msgr2=0x7f84740cd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:06.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.636+0000 7f847a1e2640 1 -- 172.21.15.98:0/969048975 shutdown_connections 2024-09-10T20:59:06.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.636+0000 7f847a1e2640 1 -- 172.21.15.98:0/969048975 wait complete. 2024-09-10T20:59:06.651 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:59:06.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.747+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/1216033959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e8406cb70 msgr2=0x7f5e840cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:06.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.747+0000 7f5e8c3cd640 1 --2- 172.21.15.98:0/1216033959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e8406cb70 0x7f5e840cb080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e84073b60 tx=0x7f5e7c02a410 comp rx=0 tx=0).stop 2024-09-10T20:59:06.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.748+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/1216033959 shutdown_connections 2024-09-10T20:59:06.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.748+0000 7f5e8c3cd640 1 --2- 172.21.15.98:0/1216033959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e8406cb70 0x7f5e840cb080 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.748+0000 7f5e8c3cd640 1 --2- 172.21.15.98:0/1216033959 >> v2:172.21.15.190:3300/0 conn(0x7f5e8406e0a0 0x7f5e840b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.748+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/1216033959 >> 172.21.15.98:0/1216033959 conn(0x7f5e84074150 msgr2=0x7f5e84074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:06.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.748+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/1216033959 shutdown_connections 2024-09-10T20:59:06.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.748+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/1216033959 wait complete. 2024-09-10T20:59:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.749+0000 7f5e8c3cd640 1 Processor -- start 2024-09-10T20:59:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.749+0000 7f5e8c3cd640 1 -- start start 2024-09-10T20:59:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.749+0000 7f5e8c3cd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e8406cb70 0x7f5e840992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.749+0000 7f5e8c3cd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e8406dcc0 0x7f5e840c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.749+0000 7f5e8c3cd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e8406e0a0 0x7f5e841f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.749+0000 7f5e8c3cd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5e840b5010 con 0x7f5e8406cb70 2024-09-10T20:59:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.749+0000 7f5e8c3cd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5e840cb5c0 con 0x7f5e8406e0a0 2024-09-10T20:59:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.749+0000 7f5e8c3cd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5e840b5190 con 0x7f5e8406dcc0 2024-09-10T20:59:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e8a169640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e8406cb70 0x7f5e840992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e8a169640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e8406cb70 0x7f5e840992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58408/0 (socket says 172.21.15.98:58408) 2024-09-10T20:59:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e8a169640 1 -- 172.21.15.98:0/3890858796 learned_addr learned my addr 172.21.15.98:0/3890858796 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e8a96a640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e8406e0a0 0x7f5e841f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e89968640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e8406dcc0 0x7f5e840c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e8a169640 1 -- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e8406dcc0 msgr2=0x7f5e840c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e8a169640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e8406dcc0 0x7f5e840c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e8a169640 1 -- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e8406e0a0 msgr2=0x7f5e841f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e8a169640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e8406e0a0 0x7f5e841f2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e8a169640 1 -- 172.21.15.98:0/3890858796 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e840c5110 con 0x7f5e8406cb70 2024-09-10T20:59:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e89968640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e8406dcc0 0x7f5e840c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e8a96a640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e8406e0a0 0x7f5e841f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e8a169640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e8406cb70 0x7f5e840992f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400c7f0 tx=0x7f5e7400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e82ffd640 1 -- 172.21.15.98:0/3890858796 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74017070 con 0x7f5e8406cb70 2024-09-10T20:59:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e82ffd640 1 -- 172.21.15.98:0/3890858796 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e74005070 con 0x7f5e8406cb70 2024-09-10T20:59:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/3890858796 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e840c53d0 con 0x7f5e8406cb70 2024-09-10T20:59:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e82ffd640 1 -- 172.21.15.98:0/3890858796 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e740054c0 con 0x7f5e8406cb70 2024-09-10T20:59:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.750+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/3890858796 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e840c5990 con 0x7f5e8406cb70 2024-09-10T20:59:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.751+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/3890858796 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e840b1720 con 0x7f5e8406cb70 2024-09-10T20:59:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.752+0000 7f5e82ffd640 1 -- 172.21.15.98:0/3890858796 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5e74005660 con 0x7f5e8406cb70 2024-09-10T20:59:06.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.752+0000 7f5e82ffd640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5e70077f70 0x7f5e7007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:06.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.753+0000 7f5e82ffd640 1 -- 172.21.15.98:0/3890858796 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5e74099b00 con 0x7f5e8406cb70 2024-09-10T20:59:06.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.753+0000 7f5e89968640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5e70077f70 0x7f5e7007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:06.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.753+0000 7f5e89968640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5e70077f70 0x7f5e7007a430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5e78005fd0 tx=0x7f5e78005e20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:06.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.757+0000 7f5e82ffd640 1 -- 172.21.15.98:0/3890858796 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5e74062240 con 0x7f5e8406cb70 2024-09-10T20:59:06.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.930+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/3890858796 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e840cc4d0 con 0x7f5e8406cb70 2024-09-10T20:59:06.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.931+0000 7f5e82ffd640 1 -- 172.21.15.98:0/3890858796 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18610 (secure 0 0 0) 0x7f5e74061b40 con 0x7f5e8406cb70 2024-09-10T20:59:06.936 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:06.936 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":25,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:58:51.602367+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:58:44.390290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T20:58:45.564099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T20:58:49.250508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:59:06.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.933+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5e70077f70 msgr2=0x7f5e7007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:06.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.933+0000 7f5e8c3cd640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5e70077f70 0x7f5e7007a430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5e78005fd0 tx=0x7f5e78005e20 comp rx=0 tx=0).stop 2024-09-10T20:59:06.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.933+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/3890858796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e8406cb70 msgr2=0x7f5e840992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:06.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.933+0000 7f5e8c3cd640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e8406cb70 0x7f5e840992f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400c7f0 tx=0x7f5e7400ccb0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.933+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/3890858796 shutdown_connections 2024-09-10T20:59:06.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.933+0000 7f5e8c3cd640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5e70077f70 0x7f5e7007a430 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.933+0000 7f5e8c3cd640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e8406e0a0 0x7f5e841f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.933+0000 7f5e8c3cd640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e8406dcc0 0x7f5e840c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.933+0000 7f5e8c3cd640 1 --2- 172.21.15.98:0/3890858796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e8406cb70 0x7f5e840992f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:06.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.933+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/3890858796 >> 172.21.15.98:0/3890858796 conn(0x7f5e84074150 msgr2=0x7f5e840cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:06.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.933+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/3890858796 shutdown_connections 2024-09-10T20:59:06.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:06.933+0000 7f5e8c3cd640 1 -- 172.21.15.98:0/3890858796 wait complete. 2024-09-10T20:59:06.949 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T20:59:06.949 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:59:07.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.046+0000 7f21f752b640 1 -- 172.21.15.98:0/3274201552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21f00b05c0 msgr2=0x7f21f00cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.046+0000 7f21f752b640 1 --2- 172.21.15.98:0/3274201552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21f00b05c0 0x7f21f00cfda0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f21e8005fd0 tx=0x7f21e802aa10 comp rx=0 tx=0).stop 2024-09-10T20:59:07.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.046+0000 7f21f752b640 1 -- 172.21.15.98:0/3274201552 shutdown_connections 2024-09-10T20:59:07.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.046+0000 7f21f752b640 1 --2- 172.21.15.98:0/3274201552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21f00b05c0 0x7f21f00cfda0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.046+0000 7f21f752b640 1 --2- 172.21.15.98:0/3274201552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21f0160530 0x7f21f00cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.046+0000 7f21f752b640 1 -- 172.21.15.98:0/3274201552 >> 172.21.15.98:0/3274201552 conn(0x7f21f00b37c0 msgr2=0x7f21f00b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:07.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.046+0000 7f21f752b640 1 -- 172.21.15.98:0/3274201552 shutdown_connections 2024-09-10T20:59:07.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.046+0000 7f21f752b640 1 -- 172.21.15.98:0/3274201552 wait complete. 2024-09-10T20:59:07.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.047+0000 7f21f752b640 1 Processor -- start 2024-09-10T20:59:07.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.047+0000 7f21f752b640 1 -- start start 2024-09-10T20:59:07.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f752b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21f00b05c0 0x7f21f00c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:07.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f752b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21f0160530 0x7f21f00cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:07.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f752b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21f00c8440 0x7f21f00cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:07.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f752b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f21f00b1990 con 0x7f21f00b05c0 2024-09-10T20:59:07.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f752b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f21f00b34f0 con 0x7f21f00c8440 2024-09-10T20:59:07.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f752b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f21f00b1b10 con 0x7f21f0160530 2024-09-10T20:59:07.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f52c7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21f00b05c0 0x7f21f00c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:07.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f52c7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21f00b05c0 0x7f21f00c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58434/0 (socket says 172.21.15.98:58434) 2024-09-10T20:59:07.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f52c7640 1 -- 172.21.15.98:0/732722683 learned_addr learned my addr 172.21.15.98:0/732722683 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:07.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f5ac8640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21f00c8440 0x7f21f00cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:07.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f4ac6640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21f0160530 0x7f21f00cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:07.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f52c7640 1 -- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21f0160530 msgr2=0x7f21f00cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f52c7640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21f0160530 0x7f21f00cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f52c7640 1 -- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21f00c8440 msgr2=0x7f21f00cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f52c7640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21f00c8440 0x7f21f00cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f52c7640 1 -- 172.21.15.98:0/732722683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f00c92a0 con 0x7f21f00b05c0 2024-09-10T20:59:07.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f5ac8640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21f00c8440 0x7f21f00cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:07.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f52c7640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21f00b05c0 0x7f21f00c8d60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00d700 tx=0x7f21ec00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:07.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21e5ffb640 1 -- 172.21.15.98:0/732722683 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec017070 con 0x7f21f00b05c0 2024-09-10T20:59:07.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21e5ffb640 1 -- 172.21.15.98:0/732722683 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21ec005310 con 0x7f21f00b05c0 2024-09-10T20:59:07.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21e5ffb640 1 -- 172.21.15.98:0/732722683 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec009820 con 0x7f21f00b05c0 2024-09-10T20:59:07.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.048+0000 7f21f752b640 1 -- 172.21.15.98:0/732722683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f00b6690 con 0x7f21f00b05c0 2024-09-10T20:59:07.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.049+0000 7f21f752b640 1 -- 172.21.15.98:0/732722683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f00b6c50 con 0x7f21f00b05c0 2024-09-10T20:59:07.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.050+0000 7f21f752b640 1 -- 172.21.15.98:0/732722683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f00b1f70 con 0x7f21f00b05c0 2024-09-10T20:59:07.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.050+0000 7f21e5ffb640 1 -- 172.21.15.98:0/732722683 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f21ec0099c0 con 0x7f21f00b05c0 2024-09-10T20:59:07.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.050+0000 7f21e5ffb640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f21d4077fa0 0x7f21d407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:07.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.051+0000 7f21e5ffb640 1 -- 172.21.15.98:0/732722683 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f21ec09a100 con 0x7f21f00b05c0 2024-09-10T20:59:07.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.051+0000 7f21f4ac6640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f21d4077fa0 0x7f21d407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:07.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.051+0000 7f21f4ac6640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f21d4077fa0 0x7f21d407a460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f21f015aa40 tx=0x7f21e0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:07.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.055+0000 7f21e5ffb640 1 -- 172.21.15.98:0/732722683 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f21ec062270 con 0x7f21f00b05c0 2024-09-10T20:59:07.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.226+0000 7f21f752b640 1 -- 172.21.15.98:0/732722683 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21f00c89c0 con 0x7f21d4077fa0 2024-09-10T20:59:07.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.232+0000 7f21e5ffb640 1 -- 172.21.15.98:0/732722683 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f21f00c89c0 con 0x7f21d4077fa0 2024-09-10T20:59:07.240 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:07.242 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-10T20:59:07.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.236+0000 7f21f752b640 1 -- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f21d4077fa0 msgr2=0x7f21d407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.236+0000 7f21f752b640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f21d4077fa0 0x7f21d407a460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f21f015aa40 tx=0x7f21e0009040 comp rx=0 tx=0).stop 2024-09-10T20:59:07.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.236+0000 7f21f752b640 1 -- 172.21.15.98:0/732722683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21f00b05c0 msgr2=0x7f21f00c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.236+0000 7f21f752b640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21f00b05c0 0x7f21f00c8d60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00d700 tx=0x7f21ec00dbc0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.236+0000 7f21f752b640 1 -- 172.21.15.98:0/732722683 shutdown_connections 2024-09-10T20:59:07.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.236+0000 7f21f752b640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f21d4077fa0 0x7f21d407a460 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.236+0000 7f21f752b640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f21f00c8440 0x7f21f00cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.236+0000 7f21f752b640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f21f0160530 0x7f21f00cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.236+0000 7f21f752b640 1 --2- 172.21.15.98:0/732722683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21f00b05c0 0x7f21f00c8d60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.236+0000 7f21f752b640 1 -- 172.21.15.98:0/732722683 >> 172.21.15.98:0/732722683 conn(0x7f21f00b37c0 msgr2=0x7f21f00aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:07.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.236+0000 7f21f752b640 1 -- 172.21.15.98:0/732722683 shutdown_connections 2024-09-10T20:59:07.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.236+0000 7f21f752b640 1 -- 172.21.15.98:0/732722683 wait complete. 2024-09-10T20:59:07.252 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-10T20:59:06.911648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3317760,"data_stored":2011640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003466"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904144+0000","last_change":"2024-09-10T20:58:46.222874+0000","last_active":"2024-09-10T20:58:53.904144+0000","last_peered":"2024-09-10T20:58:53.904144+0000","last_clean":"2024-09-10T20:58:53.904144+0000","last_became_active":"2024-09-10T20:58:46.222585+0000","last_became_peered":"2024-09-10T20:58:46.222585+0000","last_unstale":"2024-09-10T20:58:53.904144+0000","last_undegraded":"2024-09-10T20:58:53.904144+0000","last_fullsized":"2024-09-10T20:58:53.904144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:12.557183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258747+0000","last_change":"2024-09-10T20:58:47.236553+0000","last_active":"2024-09-10T20:58:52.258747+0000","last_peered":"2024-09-10T20:58:52.258747+0000","last_clean":"2024-09-10T20:58:52.258747+0000","last_became_active":"2024-09-10T20:58:47.236267+0000","last_became_peered":"2024-09-10T20:58:47.236267+0000","last_unstale":"2024-09-10T20:58:52.258747+0000","last_undegraded":"2024-09-10T20:58:52.258747+0000","last_fullsized":"2024-09-10T20:58:52.258747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:54.301892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258094+0000","last_change":"2024-09-10T20:58:51.557088+0000","last_active":"2024-09-10T20:58:52.258094+0000","last_peered":"2024-09-10T20:58:52.258094+0000","last_clean":"2024-09-10T20:58:52.258094+0000","last_became_active":"2024-09-10T20:58:51.556571+0000","last_became_peered":"2024-09-10T20:58:51.556571+0000","last_unstale":"2024-09-10T20:58:52.258094+0000","last_undegraded":"2024-09-10T20:58:52.258094+0000","last_fullsized":"2024-09-10T20:58:52.258094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:32.723152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258861+0000","last_change":"2024-09-10T20:58:46.221311+0000","last_active":"2024-09-10T20:58:52.258861+0000","last_peered":"2024-09-10T20:58:52.258861+0000","last_clean":"2024-09-10T20:58:52.258861+0000","last_became_active":"2024-09-10T20:58:46.221053+0000","last_became_peered":"2024-09-10T20:58:46.221053+0000","last_unstale":"2024-09-10T20:58:52.258861+0000","last_undegraded":"2024-09-10T20:58:52.258861+0000","last_fullsized":"2024-09-10T20:58:52.258861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:02.951532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903969+0000","last_change":"2024-09-10T20:58:47.246604+0000","last_active":"2024-09-10T20:58:53.903969+0000","last_peered":"2024-09-10T20:58:53.903969+0000","last_clean":"2024-09-10T20:58:53.903969+0000","last_became_active":"2024-09-10T20:58:47.244966+0000","last_became_peered":"2024-09-10T20:58:47.244966+0000","last_unstale":"2024-09-10T20:58:53.903969+0000","last_undegraded":"2024-09-10T20:58:53.903969+0000","last_fullsized":"2024-09-10T20:58:53.903969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:54.301892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255041+0000","last_change":"2024-09-10T20:58:51.275679+0000","last_active":"2024-09-10T20:58:52.255041+0000","last_peered":"2024-09-10T20:58:52.255041+0000","last_clean":"2024-09-10T20:58:52.255041+0000","last_became_active":"2024-09-10T20:58:51.275227+0000","last_became_peered":"2024-09-10T20:58:51.275227+0000","last_unstale":"2024-09-10T20:58:52.255041+0000","last_undegraded":"2024-09-10T20:58:52.255041+0000","last_fullsized":"2024-09-10T20:58:52.255041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:07.240517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905109+0000","last_change":"2024-09-10T20:58:46.223303+0000","last_active":"2024-09-10T20:58:53.905109+0000","last_peered":"2024-09-10T20:58:53.905109+0000","last_clean":"2024-09-10T20:58:53.905109+0000","last_became_active":"2024-09-10T20:58:46.222924+0000","last_became_peered":"2024-09-10T20:58:46.222924+0000","last_unstale":"2024-09-10T20:58:53.905109+0000","last_undegraded":"2024-09-10T20:58:53.905109+0000","last_fullsized":"2024-09-10T20:58:53.905109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:37.987408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255679+0000","last_change":"2024-09-10T20:58:47.231063+0000","last_active":"2024-09-10T20:58:52.255679+0000","last_peered":"2024-09-10T20:58:52.255679+0000","last_clean":"2024-09-10T20:58:52.255679+0000","last_became_active":"2024-09-10T20:58:47.230764+0000","last_became_peered":"2024-09-10T20:58:47.230764+0000","last_unstale":"2024-09-10T20:58:52.255679+0000","last_undegraded":"2024-09-10T20:58:52.255679+0000","last_fullsized":"2024-09-10T20:58:52.255679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:17.763154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256830+0000","last_change":"2024-09-10T20:58:51.275019+0000","last_active":"2024-09-10T20:58:52.256830+0000","last_peered":"2024-09-10T20:58:52.256830+0000","last_clean":"2024-09-10T20:58:52.256830+0000","last_became_active":"2024-09-10T20:58:51.274298+0000","last_became_peered":"2024-09-10T20:58:51.274298+0000","last_unstale":"2024-09-10T20:58:52.256830+0000","last_undegraded":"2024-09-10T20:58:52.256830+0000","last_fullsized":"2024-09-10T20:58:52.256830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.486482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904772+0000","last_change":"2024-09-10T20:58:46.231109+0000","last_active":"2024-09-10T20:58:53.904772+0000","last_peered":"2024-09-10T20:58:53.904772+0000","last_clean":"2024-09-10T20:58:53.904772+0000","last_became_active":"2024-09-10T20:58:46.230798+0000","last_became_peered":"2024-09-10T20:58:46.230798+0000","last_unstale":"2024-09-10T20:58:53.904772+0000","last_undegraded":"2024-09-10T20:58:53.904772+0000","last_fullsized":"2024-09-10T20:58:53.904772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:31.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256327+0000","last_change":"2024-09-10T20:58:47.230225+0000","last_active":"2024-09-10T20:58:52.256327+0000","last_peered":"2024-09-10T20:58:52.256327+0000","last_clean":"2024-09-10T20:58:52.256327+0000","last_became_active":"2024-09-10T20:58:47.229943+0000","last_became_peered":"2024-09-10T20:58:47.229943+0000","last_unstale":"2024-09-10T20:58:52.256327+0000","last_undegraded":"2024-09-10T20:58:52.256327+0000","last_fullsized":"2024-09-10T20:58:52.256327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:45.849686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256190+0000","last_change":"2024-09-10T20:58:51.270577+0000","last_active":"2024-09-10T20:58:52.256190+0000","last_peered":"2024-09-10T20:58:52.256190+0000","last_clean":"2024-09-10T20:58:52.256190+0000","last_became_active":"2024-09-10T20:58:51.270223+0000","last_became_peered":"2024-09-10T20:58:51.270223+0000","last_unstale":"2024-09-10T20:58:52.256190+0000","last_undegraded":"2024-09-10T20:58:52.256190+0000","last_fullsized":"2024-09-10T20:58:52.256190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:45.149805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904189+0000","last_change":"2024-09-10T20:58:47.230068+0000","last_active":"2024-09-10T20:58:53.904189+0000","last_peered":"2024-09-10T20:58:53.904189+0000","last_clean":"2024-09-10T20:58:53.904189+0000","last_became_active":"2024-09-10T20:58:47.229755+0000","last_became_peered":"2024-09-10T20:58:47.229755+0000","last_unstale":"2024-09-10T20:58:53.904189+0000","last_undegraded":"2024-09-10T20:58:53.904189+0000","last_fullsized":"2024-09-10T20:58:53.904189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:04.498018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904966+0000","last_change":"2024-09-10T20:58:46.227149+0000","last_active":"2024-09-10T20:58:53.904966+0000","last_peered":"2024-09-10T20:58:53.904966+0000","last_clean":"2024-09-10T20:58:53.904966+0000","last_became_active":"2024-09-10T20:58:46.226863+0000","last_became_peered":"2024-09-10T20:58:46.226863+0000","last_unstale":"2024-09-10T20:58:53.904966+0000","last_undegraded":"2024-09-10T20:58:53.904966+0000","last_fullsized":"2024-09-10T20:58:53.904966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:28.108849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256521+0000","last_change":"2024-09-10T20:58:51.280288+0000","last_active":"2024-09-10T20:58:52.256521+0000","last_peered":"2024-09-10T20:58:52.256521+0000","last_clean":"2024-09-10T20:58:52.256521+0000","last_became_active":"2024-09-10T20:58:51.279918+0000","last_became_peered":"2024-09-10T20:58:51.279918+0000","last_unstale":"2024-09-10T20:58:52.256521+0000","last_undegraded":"2024-09-10T20:58:52.256521+0000","last_fullsized":"2024-09-10T20:58:52.256521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:36.307037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258303+0000","last_change":"2024-09-10T20:58:47.246700+0000","last_active":"2024-09-10T20:58:52.258303+0000","last_peered":"2024-09-10T20:58:52.258303+0000","last_clean":"2024-09-10T20:58:52.258303+0000","last_became_active":"2024-09-10T20:58:47.246456+0000","last_became_peered":"2024-09-10T20:58:47.246456+0000","last_unstale":"2024-09-10T20:58:52.258303+0000","last_undegraded":"2024-09-10T20:58:52.258303+0000","last_fullsized":"2024-09-10T20:58:52.258303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:08.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316714+0000","last_change":"2024-09-10T20:58:46.224904+0000","last_active":"2024-09-10T20:58:53.316714+0000","last_peered":"2024-09-10T20:58:53.316714+0000","last_clean":"2024-09-10T20:58:53.316714+0000","last_became_active":"2024-09-10T20:58:46.224619+0000","last_became_peered":"2024-09-10T20:58:46.224619+0000","last_unstale":"2024-09-10T20:58:53.316714+0000","last_undegraded":"2024-09-10T20:58:53.316714+0000","last_fullsized":"2024-09-10T20:58:53.316714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:34.223018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256285+0000","last_change":"2024-09-10T20:58:51.274096+0000","last_active":"2024-09-10T20:58:52.256285+0000","last_peered":"2024-09-10T20:58:52.256285+0000","last_clean":"2024-09-10T20:58:52.256285+0000","last_became_active":"2024-09-10T20:58:51.273636+0000","last_became_peered":"2024-09-10T20:58:51.273636+0000","last_unstale":"2024-09-10T20:58:52.256285+0000","last_undegraded":"2024-09-10T20:58:52.256285+0000","last_fullsized":"2024-09-10T20:58:52.256285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:05.160808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258649+0000","last_change":"2024-09-10T20:58:47.228381+0000","last_active":"2024-09-10T20:58:52.258649+0000","last_peered":"2024-09-10T20:58:52.258649+0000","last_clean":"2024-09-10T20:58:52.258649+0000","last_became_active":"2024-09-10T20:58:47.228080+0000","last_became_peered":"2024-09-10T20:58:47.228080+0000","last_unstale":"2024-09-10T20:58:52.258649+0000","last_undegraded":"2024-09-10T20:58:52.258649+0000","last_fullsized":"2024-09-10T20:58:52.258649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:39.726666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316453+0000","last_change":"2024-09-10T20:58:46.226903+0000","last_active":"2024-09-10T20:58:53.316453+0000","last_peered":"2024-09-10T20:58:53.316453+0000","last_clean":"2024-09-10T20:58:53.316453+0000","last_became_active":"2024-09-10T20:58:46.226649+0000","last_became_peered":"2024-09-10T20:58:46.226649+0000","last_unstale":"2024-09-10T20:58:53.316453+0000","last_undegraded":"2024-09-10T20:58:53.316453+0000","last_fullsized":"2024-09-10T20:58:53.316453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:14.748746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317126+0000","last_change":"2024-09-10T20:58:51.554471+0000","last_active":"2024-09-10T20:58:53.317126+0000","last_peered":"2024-09-10T20:58:53.317126+0000","last_clean":"2024-09-10T20:58:53.317126+0000","last_became_active":"2024-09-10T20:58:51.554080+0000","last_became_peered":"2024-09-10T20:58:51.554080+0000","last_unstale":"2024-09-10T20:58:53.317126+0000","last_undegraded":"2024-09-10T20:58:53.317126+0000","last_fullsized":"2024-09-10T20:58:53.317126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:24.547698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256775+0000","last_change":"2024-09-10T20:58:47.234236+0000","last_active":"2024-09-10T20:58:52.256775+0000","last_peered":"2024-09-10T20:58:52.256775+0000","last_clean":"2024-09-10T20:58:52.256775+0000","last_became_active":"2024-09-10T20:58:47.233924+0000","last_became_peered":"2024-09-10T20:58:47.233924+0000","last_unstale":"2024-09-10T20:58:52.256775+0000","last_undegraded":"2024-09-10T20:58:52.256775+0000","last_fullsized":"2024-09-10T20:58:52.256775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:38.992405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903782+0000","last_change":"2024-09-10T20:58:46.221112+0000","last_active":"2024-09-10T20:58:53.903782+0000","last_peered":"2024-09-10T20:58:53.903782+0000","last_clean":"2024-09-10T20:58:53.903782+0000","last_became_active":"2024-09-10T20:58:46.220837+0000","last_became_peered":"2024-09-10T20:58:46.220837+0000","last_unstale":"2024-09-10T20:58:53.903782+0000","last_undegraded":"2024-09-10T20:58:53.903782+0000","last_fullsized":"2024-09-10T20:58:53.903782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:27.129152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256427+0000","last_change":"2024-09-10T20:58:51.274965+0000","last_active":"2024-09-10T20:58:52.256427+0000","last_peered":"2024-09-10T20:58:52.256427+0000","last_clean":"2024-09-10T20:58:52.256427+0000","last_became_active":"2024-09-10T20:58:51.274557+0000","last_became_peered":"2024-09-10T20:58:51.274557+0000","last_unstale":"2024-09-10T20:58:52.256427+0000","last_undegraded":"2024-09-10T20:58:52.256427+0000","last_fullsized":"2024-09-10T20:58:52.256427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:16.787797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904100+0000","last_change":"2024-09-10T20:58:47.246893+0000","last_active":"2024-09-10T20:58:53.904100+0000","last_peered":"2024-09-10T20:58:53.904100+0000","last_clean":"2024-09-10T20:58:53.904100+0000","last_became_active":"2024-09-10T20:58:47.245226+0000","last_became_peered":"2024-09-10T20:58:47.245226+0000","last_unstale":"2024-09-10T20:58:53.904100+0000","last_undegraded":"2024-09-10T20:58:53.904100+0000","last_fullsized":"2024-09-10T20:58:53.904100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:58.155371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904645+0000","last_change":"2024-09-10T20:58:46.222772+0000","last_active":"2024-09-10T20:58:53.904645+0000","last_peered":"2024-09-10T20:58:53.904645+0000","last_clean":"2024-09-10T20:58:53.904645+0000","last_became_active":"2024-09-10T20:58:46.222461+0000","last_became_peered":"2024-09-10T20:58:46.222461+0000","last_unstale":"2024-09-10T20:58:53.904645+0000","last_undegraded":"2024-09-10T20:58:53.904645+0000","last_fullsized":"2024-09-10T20:58:53.904645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:28.108849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256282+0000","last_change":"2024-09-10T20:58:51.554529+0000","last_active":"2024-09-10T20:58:52.256282+0000","last_peered":"2024-09-10T20:58:52.256282+0000","last_clean":"2024-09-10T20:58:52.256282+0000","last_became_active":"2024-09-10T20:58:51.554104+0000","last_became_peered":"2024-09-10T20:58:51.554104+0000","last_unstale":"2024-09-10T20:58:52.256282+0000","last_undegraded":"2024-09-10T20:58:52.256282+0000","last_fullsized":"2024-09-10T20:58:52.256282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:18.603248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.259179+0000","last_change":"2024-09-10T20:58:47.249780+0000","last_active":"2024-09-10T20:58:52.259179+0000","last_peered":"2024-09-10T20:58:52.259179+0000","last_clean":"2024-09-10T20:58:52.259179+0000","last_became_active":"2024-09-10T20:58:47.249519+0000","last_became_peered":"2024-09-10T20:58:47.249519+0000","last_unstale":"2024-09-10T20:58:52.259179+0000","last_undegraded":"2024-09-10T20:58:52.259179+0000","last_fullsized":"2024-09-10T20:58:52.259179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:58.169931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316888+0000","last_change":"2024-09-10T20:58:46.225535+0000","last_active":"2024-09-10T20:58:53.316888+0000","last_peered":"2024-09-10T20:58:53.316888+0000","last_clean":"2024-09-10T20:58:53.316888+0000","last_became_active":"2024-09-10T20:58:46.225248+0000","last_became_peered":"2024-09-10T20:58:46.225248+0000","last_unstale":"2024-09-10T20:58:53.316888+0000","last_undegraded":"2024-09-10T20:58:53.316888+0000","last_fullsized":"2024-09-10T20:58:53.316888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:22.880233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258298+0000","last_change":"2024-09-10T20:58:51.559242+0000","last_active":"2024-09-10T20:58:52.258298+0000","last_peered":"2024-09-10T20:58:52.258298+0000","last_clean":"2024-09-10T20:58:52.258298+0000","last_became_active":"2024-09-10T20:58:51.558875+0000","last_became_peered":"2024-09-10T20:58:51.558875+0000","last_unstale":"2024-09-10T20:58:52.258298+0000","last_undegraded":"2024-09-10T20:58:52.258298+0000","last_fullsized":"2024-09-10T20:58:52.258298+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:18.246776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317045+0000","last_change":"2024-09-10T20:58:47.228376+0000","last_active":"2024-09-10T20:58:53.317045+0000","last_peered":"2024-09-10T20:58:53.317045+0000","last_clean":"2024-09-10T20:58:53.317045+0000","last_became_active":"2024-09-10T20:58:47.228086+0000","last_became_peered":"2024-09-10T20:58:47.228086+0000","last_unstale":"2024-09-10T20:58:53.317045+0000","last_undegraded":"2024-09-10T20:58:53.317045+0000","last_fullsized":"2024-09-10T20:58:53.317045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:07.109711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905238+0000","last_change":"2024-09-10T20:58:46.221847+0000","last_active":"2024-09-10T20:58:53.905238+0000","last_peered":"2024-09-10T20:58:53.905238+0000","last_clean":"2024-09-10T20:58:53.905238+0000","last_became_active":"2024-09-10T20:58:46.221571+0000","last_became_peered":"2024-09-10T20:58:46.221571+0000","last_unstale":"2024-09-10T20:58:53.905238+0000","last_undegraded":"2024-09-10T20:58:53.905238+0000","last_fullsized":"2024-09-10T20:58:53.905238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:44.242563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903819+0000","last_change":"2024-09-10T20:58:51.554547+0000","last_active":"2024-09-10T20:58:53.903819+0000","last_peered":"2024-09-10T20:58:53.903819+0000","last_clean":"2024-09-10T20:58:53.903819+0000","last_became_active":"2024-09-10T20:58:51.554197+0000","last_became_peered":"2024-09-10T20:58:51.554197+0000","last_unstale":"2024-09-10T20:58:53.903819+0000","last_undegraded":"2024-09-10T20:58:53.903819+0000","last_fullsized":"2024-09-10T20:58:53.903819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:25.499034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256428+0000","last_change":"2024-09-10T20:58:47.243797+0000","last_active":"2024-09-10T20:58:52.256428+0000","last_peered":"2024-09-10T20:58:52.256428+0000","last_clean":"2024-09-10T20:58:52.256428+0000","last_became_active":"2024-09-10T20:58:47.243539+0000","last_became_peered":"2024-09-10T20:58:47.243539+0000","last_unstale":"2024-09-10T20:58:52.256428+0000","last_undegraded":"2024-09-10T20:58:52.256428+0000","last_fullsized":"2024-09-10T20:58:52.256428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:30.226963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317072+0000","last_change":"2024-09-10T20:58:46.221373+0000","last_active":"2024-09-10T20:58:53.317072+0000","last_peered":"2024-09-10T20:58:53.317072+0000","last_clean":"2024-09-10T20:58:53.317072+0000","last_became_active":"2024-09-10T20:58:46.221104+0000","last_became_peered":"2024-09-10T20:58:46.221104+0000","last_unstale":"2024-09-10T20:58:53.317072+0000","last_undegraded":"2024-09-10T20:58:53.317072+0000","last_fullsized":"2024-09-10T20:58:53.317072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:29.110234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316552+0000","last_change":"2024-09-10T20:58:51.278666+0000","last_active":"2024-09-10T20:58:53.316552+0000","last_peered":"2024-09-10T20:58:53.316552+0000","last_clean":"2024-09-10T20:58:53.316552+0000","last_became_active":"2024-09-10T20:58:51.278033+0000","last_became_peered":"2024-09-10T20:58:51.278033+0000","last_unstale":"2024-09-10T20:58:53.316552+0000","last_undegraded":"2024-09-10T20:58:53.316552+0000","last_fullsized":"2024-09-10T20:58:53.316552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:59.252455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257022+0000","last_change":"2024-09-10T20:58:47.246884+0000","last_active":"2024-09-10T20:58:52.257022+0000","last_peered":"2024-09-10T20:58:52.257022+0000","last_clean":"2024-09-10T20:58:52.257022+0000","last_became_active":"2024-09-10T20:58:47.246617+0000","last_became_peered":"2024-09-10T20:58:47.246617+0000","last_unstale":"2024-09-10T20:58:52.257022+0000","last_undegraded":"2024-09-10T20:58:52.257022+0000","last_fullsized":"2024-09-10T20:58:52.257022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:32.318955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256070+0000","last_change":"2024-09-10T20:58:46.225503+0000","last_active":"2024-09-10T20:58:52.256070+0000","last_peered":"2024-09-10T20:58:52.256070+0000","last_clean":"2024-09-10T20:58:52.256070+0000","last_became_active":"2024-09-10T20:58:46.225088+0000","last_became_peered":"2024-09-10T20:58:46.225088+0000","last_unstale":"2024-09-10T20:58:52.256070+0000","last_undegraded":"2024-09-10T20:58:52.256070+0000","last_fullsized":"2024-09-10T20:58:52.256070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:36.394527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317025+0000","last_change":"2024-09-10T20:58:51.277749+0000","last_active":"2024-09-10T20:58:53.317025+0000","last_peered":"2024-09-10T20:58:53.317025+0000","last_clean":"2024-09-10T20:58:53.317025+0000","last_became_active":"2024-09-10T20:58:51.277142+0000","last_became_peered":"2024-09-10T20:58:51.277142+0000","last_unstale":"2024-09-10T20:58:53.317025+0000","last_undegraded":"2024-09-10T20:58:53.317025+0000","last_fullsized":"2024-09-10T20:58:53.317025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:42.740546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316380+0000","last_change":"2024-09-10T20:58:47.228660+0000","last_active":"2024-09-10T20:58:53.316380+0000","last_peered":"2024-09-10T20:58:53.316380+0000","last_clean":"2024-09-10T20:58:53.316380+0000","last_became_active":"2024-09-10T20:58:47.228218+0000","last_became_peered":"2024-09-10T20:58:47.228218+0000","last_unstale":"2024-09-10T20:58:53.316380+0000","last_undegraded":"2024-09-10T20:58:53.316380+0000","last_fullsized":"2024-09-10T20:58:53.316380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:51.280929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903422+0000","last_change":"2024-09-10T20:58:46.221228+0000","last_active":"2024-09-10T20:58:53.903422+0000","last_peered":"2024-09-10T20:58:53.903422+0000","last_clean":"2024-09-10T20:58:53.903422+0000","last_became_active":"2024-09-10T20:58:46.220898+0000","last_became_peered":"2024-09-10T20:58:46.220898+0000","last_unstale":"2024-09-10T20:58:53.903422+0000","last_undegraded":"2024-09-10T20:58:53.903422+0000","last_fullsized":"2024-09-10T20:58:53.903422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:18.355281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317144+0000","last_change":"2024-09-10T20:58:51.274949+0000","last_active":"2024-09-10T20:58:53.317144+0000","last_peered":"2024-09-10T20:58:53.317144+0000","last_clean":"2024-09-10T20:58:53.317144+0000","last_became_active":"2024-09-10T20:58:51.274579+0000","last_became_peered":"2024-09-10T20:58:51.274579+0000","last_unstale":"2024-09-10T20:58:53.317144+0000","last_undegraded":"2024-09-10T20:58:53.317144+0000","last_fullsized":"2024-09-10T20:58:53.317144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:50.176038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903242+0000","last_change":"2024-09-10T20:58:47.240914+0000","last_active":"2024-09-10T20:58:53.903242+0000","last_peered":"2024-09-10T20:58:53.903242+0000","last_clean":"2024-09-10T20:58:53.903242+0000","last_became_active":"2024-09-10T20:58:47.240250+0000","last_became_peered":"2024-09-10T20:58:47.240250+0000","last_unstale":"2024-09-10T20:58:53.903242+0000","last_undegraded":"2024-09-10T20:58:53.903242+0000","last_fullsized":"2024-09-10T20:58:53.903242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:46.184593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903948+0000","last_change":"2024-09-10T20:58:46.224644+0000","last_active":"2024-09-10T20:58:53.903948+0000","last_peered":"2024-09-10T20:58:53.903948+0000","last_clean":"2024-09-10T20:58:53.903948+0000","last_became_active":"2024-09-10T20:58:46.224329+0000","last_became_peered":"2024-09-10T20:58:46.224329+0000","last_unstale":"2024-09-10T20:58:53.903948+0000","last_undegraded":"2024-09-10T20:58:53.903948+0000","last_fullsized":"2024-09-10T20:58:53.903948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:01.590603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904042+0000","last_change":"2024-09-10T20:58:51.554468+0000","last_active":"2024-09-10T20:58:53.904042+0000","last_peered":"2024-09-10T20:58:53.904042+0000","last_clean":"2024-09-10T20:58:53.904042+0000","last_became_active":"2024-09-10T20:58:51.554092+0000","last_became_peered":"2024-09-10T20:58:51.554092+0000","last_unstale":"2024-09-10T20:58:53.904042+0000","last_undegraded":"2024-09-10T20:58:53.904042+0000","last_fullsized":"2024-09-10T20:58:53.904042+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:48.958938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258417+0000","last_change":"2024-09-10T20:58:47.242758+0000","last_active":"2024-09-10T20:58:52.258417+0000","last_peered":"2024-09-10T20:58:52.258417+0000","last_clean":"2024-09-10T20:58:52.258417+0000","last_became_active":"2024-09-10T20:58:47.242439+0000","last_became_peered":"2024-09-10T20:58:47.242439+0000","last_unstale":"2024-09-10T20:58:52.258417+0000","last_undegraded":"2024-09-10T20:58:52.258417+0000","last_fullsized":"2024-09-10T20:58:52.258417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:45.212911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255997+0000","last_change":"2024-09-10T20:58:46.222264+0000","last_active":"2024-09-10T20:58:52.255997+0000","last_peered":"2024-09-10T20:58:52.255997+0000","last_clean":"2024-09-10T20:58:52.255997+0000","last_became_active":"2024-09-10T20:58:46.221414+0000","last_became_peered":"2024-09-10T20:58:46.221414+0000","last_unstale":"2024-09-10T20:58:52.255997+0000","last_undegraded":"2024-09-10T20:58:52.255997+0000","last_fullsized":"2024-09-10T20:58:52.255997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:22.019957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":119,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904053+0000","last_change":"2024-09-10T20:58:36.895993+0000","last_active":"2024-09-10T20:58:53.904053+0000","last_peered":"2024-09-10T20:58:53.904053+0000","last_clean":"2024-09-10T20:58:53.904053+0000","last_became_active":"2024-09-10T20:58:36.894988+0000","last_became_peered":"2024-09-10T20:58:36.894988+0000","last_unstale":"2024-09-10T20:58:53.904053+0000","last_undegraded":"2024-09-10T20:58:53.904053+0000","last_fullsized":"2024-09-10T20:58:53.904053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_clean_scrub_stamp":"2024-09-10T20:58:35.588702+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.082528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904143+0000","last_change":"2024-09-10T20:58:51.556599+0000","last_active":"2024-09-10T20:58:53.904143+0000","last_peered":"2024-09-10T20:58:53.904143+0000","last_clean":"2024-09-10T20:58:53.904143+0000","last_became_active":"2024-09-10T20:58:51.556210+0000","last_became_peered":"2024-09-10T20:58:51.556210+0000","last_unstale":"2024-09-10T20:58:53.904143+0000","last_undegraded":"2024-09-10T20:58:53.904143+0000","last_fullsized":"2024-09-10T20:58:53.904143+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:36.307037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904986+0000","last_change":"2024-09-10T20:58:47.245638+0000","last_active":"2024-09-10T20:58:53.904986+0000","last_peered":"2024-09-10T20:58:53.904986+0000","last_clean":"2024-09-10T20:58:53.904986+0000","last_became_active":"2024-09-10T20:58:47.244988+0000","last_became_peered":"2024-09-10T20:58:47.244988+0000","last_unstale":"2024-09-10T20:58:53.904986+0000","last_undegraded":"2024-09-10T20:58:53.904986+0000","last_fullsized":"2024-09-10T20:58:53.904986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:04.445410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903834+0000","last_change":"2024-09-10T20:58:46.228090+0000","last_active":"2024-09-10T20:58:53.903834+0000","last_peered":"2024-09-10T20:58:53.903834+0000","last_clean":"2024-09-10T20:58:53.903834+0000","last_became_active":"2024-09-10T20:58:46.227789+0000","last_became_peered":"2024-09-10T20:58:46.227789+0000","last_unstale":"2024-09-10T20:58:53.903834+0000","last_undegraded":"2024-09-10T20:58:53.903834+0000","last_fullsized":"2024-09-10T20:58:53.903834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:37.064288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904078+0000","last_change":"2024-09-10T20:58:51.554553+0000","last_active":"2024-09-10T20:58:53.904078+0000","last_peered":"2024-09-10T20:58:53.904078+0000","last_clean":"2024-09-10T20:58:53.904078+0000","last_became_active":"2024-09-10T20:58:51.554131+0000","last_became_peered":"2024-09-10T20:58:51.554131+0000","last_unstale":"2024-09-10T20:58:53.904078+0000","last_undegraded":"2024-09-10T20:58:53.904078+0000","last_fullsized":"2024-09-10T20:58:53.904078+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:18.603248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255727+0000","last_change":"2024-09-10T20:58:47.245802+0000","last_active":"2024-09-10T20:58:52.255727+0000","last_peered":"2024-09-10T20:58:52.255727+0000","last_clean":"2024-09-10T20:58:52.255727+0000","last_became_active":"2024-09-10T20:58:47.245543+0000","last_became_peered":"2024-09-10T20:58:47.245543+0000","last_unstale":"2024-09-10T20:58:52.255727+0000","last_undegraded":"2024-09-10T20:58:52.255727+0000","last_fullsized":"2024-09-10T20:58:52.255727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:17.787548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316844+0000","last_change":"2024-09-10T20:58:46.226841+0000","last_active":"2024-09-10T20:58:53.316844+0000","last_peered":"2024-09-10T20:58:53.316844+0000","last_clean":"2024-09-10T20:58:53.316844+0000","last_became_active":"2024-09-10T20:58:46.226610+0000","last_became_peered":"2024-09-10T20:58:46.226610+0000","last_unstale":"2024-09-10T20:58:53.316844+0000","last_undegraded":"2024-09-10T20:58:53.316844+0000","last_fullsized":"2024-09-10T20:58:53.316844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:41.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904125+0000","last_change":"2024-09-10T20:58:51.274667+0000","last_active":"2024-09-10T20:58:53.904125+0000","last_peered":"2024-09-10T20:58:53.904125+0000","last_clean":"2024-09-10T20:58:53.904125+0000","last_became_active":"2024-09-10T20:58:51.274284+0000","last_became_peered":"2024-09-10T20:58:51.274284+0000","last_unstale":"2024-09-10T20:58:53.904125+0000","last_undegraded":"2024-09-10T20:58:53.904125+0000","last_fullsized":"2024-09-10T20:58:53.904125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:36.498874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905137+0000","last_change":"2024-09-10T20:58:47.245680+0000","last_active":"2024-09-10T20:58:53.905137+0000","last_peered":"2024-09-10T20:58:53.905137+0000","last_clean":"2024-09-10T20:58:53.905137+0000","last_became_active":"2024-09-10T20:58:47.245058+0000","last_became_peered":"2024-09-10T20:58:47.245058+0000","last_unstale":"2024-09-10T20:58:53.905137+0000","last_undegraded":"2024-09-10T20:58:53.905137+0000","last_fullsized":"2024-09-10T20:58:53.905137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:36.576513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903742+0000","last_change":"2024-09-10T20:58:46.222762+0000","last_active":"2024-09-10T20:58:53.903742+0000","last_peered":"2024-09-10T20:58:53.903742+0000","last_clean":"2024-09-10T20:58:53.903742+0000","last_became_active":"2024-09-10T20:58:46.222478+0000","last_became_peered":"2024-09-10T20:58:46.222478+0000","last_unstale":"2024-09-10T20:58:53.903742+0000","last_undegraded":"2024-09-10T20:58:53.903742+0000","last_fullsized":"2024-09-10T20:58:53.903742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:20.646322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256423+0000","last_change":"2024-09-10T20:58:51.554839+0000","last_active":"2024-09-10T20:58:52.256423+0000","last_peered":"2024-09-10T20:58:52.256423+0000","last_clean":"2024-09-10T20:58:52.256423+0000","last_became_active":"2024-09-10T20:58:51.554371+0000","last_became_peered":"2024-09-10T20:58:51.554371+0000","last_unstale":"2024-09-10T20:58:52.256423+0000","last_undegraded":"2024-09-10T20:58:52.256423+0000","last_fullsized":"2024-09-10T20:58:52.256423+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:16.242767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905199+0000","last_change":"2024-09-10T20:58:47.239517+0000","last_active":"2024-09-10T20:58:53.905199+0000","last_peered":"2024-09-10T20:58:53.905199+0000","last_clean":"2024-09-10T20:58:53.905199+0000","last_became_active":"2024-09-10T20:58:47.239275+0000","last_became_peered":"2024-09-10T20:58:47.239275+0000","last_unstale":"2024-09-10T20:58:53.905199+0000","last_undegraded":"2024-09-10T20:58:53.905199+0000","last_fullsized":"2024-09-10T20:58:53.905199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:27.143705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256315+0000","last_change":"2024-09-10T20:58:46.226170+0000","last_active":"2024-09-10T20:58:52.256315+0000","last_peered":"2024-09-10T20:58:52.256315+0000","last_clean":"2024-09-10T20:58:52.256315+0000","last_became_active":"2024-09-10T20:58:46.225888+0000","last_became_peered":"2024-09-10T20:58:46.225888+0000","last_unstale":"2024-09-10T20:58:52.256315+0000","last_undegraded":"2024-09-10T20:58:52.256315+0000","last_fullsized":"2024-09-10T20:58:52.256315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:27.108377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255461+0000","last_change":"2024-09-10T20:58:51.276340+0000","last_active":"2024-09-10T20:58:52.255461+0000","last_peered":"2024-09-10T20:58:52.255461+0000","last_clean":"2024-09-10T20:58:52.255461+0000","last_became_active":"2024-09-10T20:58:51.275903+0000","last_became_peered":"2024-09-10T20:58:51.275903+0000","last_unstale":"2024-09-10T20:58:52.255461+0000","last_undegraded":"2024-09-10T20:58:52.255461+0000","last_fullsized":"2024-09-10T20:58:52.255461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:04.361773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904252+0000","last_change":"2024-09-10T20:58:47.237276+0000","last_active":"2024-09-10T20:58:53.904252+0000","last_peered":"2024-09-10T20:58:53.904252+0000","last_clean":"2024-09-10T20:58:53.904252+0000","last_became_active":"2024-09-10T20:58:47.235334+0000","last_became_peered":"2024-09-10T20:58:47.235334+0000","last_unstale":"2024-09-10T20:58:53.904252+0000","last_undegraded":"2024-09-10T20:58:53.904252+0000","last_fullsized":"2024-09-10T20:58:53.904252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:58.169931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903190+0000","last_change":"2024-09-10T20:58:46.224350+0000","last_active":"2024-09-10T20:58:53.903190+0000","last_peered":"2024-09-10T20:58:53.903190+0000","last_clean":"2024-09-10T20:58:53.903190+0000","last_became_active":"2024-09-10T20:58:46.223931+0000","last_became_peered":"2024-09-10T20:58:46.223931+0000","last_unstale":"2024-09-10T20:58:53.903190+0000","last_undegraded":"2024-09-10T20:58:53.903190+0000","last_fullsized":"2024-09-10T20:58:53.903190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:22.019957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903918+0000","last_change":"2024-09-10T20:58:51.270969+0000","last_active":"2024-09-10T20:58:53.903918+0000","last_peered":"2024-09-10T20:58:53.903918+0000","last_clean":"2024-09-10T20:58:53.903918+0000","last_became_active":"2024-09-10T20:58:51.270229+0000","last_became_peered":"2024-09-10T20:58:51.270229+0000","last_unstale":"2024-09-10T20:58:53.903918+0000","last_undegraded":"2024-09-10T20:58:53.903918+0000","last_fullsized":"2024-09-10T20:58:53.903918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:35.803700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903940+0000","last_change":"2024-09-10T20:58:47.247096+0000","last_active":"2024-09-10T20:58:53.903940+0000","last_peered":"2024-09-10T20:58:53.903940+0000","last_clean":"2024-09-10T20:58:53.903940+0000","last_became_active":"2024-09-10T20:58:47.246797+0000","last_became_peered":"2024-09-10T20:58:47.246797+0000","last_unstale":"2024-09-10T20:58:53.903940+0000","last_undegraded":"2024-09-10T20:58:53.903940+0000","last_fullsized":"2024-09-10T20:58:53.903940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:18.950297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904318+0000","last_change":"2024-09-10T20:58:46.226890+0000","last_active":"2024-09-10T20:58:53.904318+0000","last_peered":"2024-09-10T20:58:53.904318+0000","last_clean":"2024-09-10T20:58:53.904318+0000","last_became_active":"2024-09-10T20:58:46.226615+0000","last_became_peered":"2024-09-10T20:58:46.226615+0000","last_unstale":"2024-09-10T20:58:53.904318+0000","last_undegraded":"2024-09-10T20:58:53.904318+0000","last_fullsized":"2024-09-10T20:58:53.904318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:08.671158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.259085+0000","last_change":"2024-09-10T20:58:51.556517+0000","last_active":"2024-09-10T20:58:52.259085+0000","last_peered":"2024-09-10T20:58:52.259085+0000","last_clean":"2024-09-10T20:58:52.259085+0000","last_became_active":"2024-09-10T20:58:51.556152+0000","last_became_peered":"2024-09-10T20:58:51.556152+0000","last_unstale":"2024-09-10T20:58:52.259085+0000","last_undegraded":"2024-09-10T20:58:52.259085+0000","last_fullsized":"2024-09-10T20:58:52.259085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:46.913894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904720+0000","last_change":"2024-09-10T20:58:47.245297+0000","last_active":"2024-09-10T20:58:53.904720+0000","last_peered":"2024-09-10T20:58:53.904720+0000","last_clean":"2024-09-10T20:58:53.904720+0000","last_became_active":"2024-09-10T20:58:47.244947+0000","last_became_peered":"2024-09-10T20:58:47.244947+0000","last_unstale":"2024-09-10T20:58:53.904720+0000","last_undegraded":"2024-09-10T20:58:53.904720+0000","last_fullsized":"2024-09-10T20:58:53.904720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:25.115061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255795+0000","last_change":"2024-09-10T20:58:46.229169+0000","last_active":"2024-09-10T20:58:52.255795+0000","last_peered":"2024-09-10T20:58:52.255795+0000","last_clean":"2024-09-10T20:58:52.255795+0000","last_became_active":"2024-09-10T20:58:46.228622+0000","last_became_peered":"2024-09-10T20:58:46.228622+0000","last_unstale":"2024-09-10T20:58:52.255795+0000","last_undegraded":"2024-09-10T20:58:52.255795+0000","last_fullsized":"2024-09-10T20:58:52.255795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:35.810342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258536+0000","last_change":"2024-09-10T20:58:51.554436+0000","last_active":"2024-09-10T20:58:52.258536+0000","last_peered":"2024-09-10T20:58:52.258536+0000","last_clean":"2024-09-10T20:58:52.258536+0000","last_became_active":"2024-09-10T20:58:51.554038+0000","last_became_peered":"2024-09-10T20:58:51.554038+0000","last_unstale":"2024-09-10T20:58:52.258536+0000","last_undegraded":"2024-09-10T20:58:52.258536+0000","last_fullsized":"2024-09-10T20:58:52.258536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:33.352140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903833+0000","last_change":"2024-09-10T20:58:47.247094+0000","last_active":"2024-09-10T20:58:53.903833+0000","last_peered":"2024-09-10T20:58:53.903833+0000","last_clean":"2024-09-10T20:58:53.903833+0000","last_became_active":"2024-09-10T20:58:47.246095+0000","last_became_peered":"2024-09-10T20:58:47.246095+0000","last_unstale":"2024-09-10T20:58:53.903833+0000","last_undegraded":"2024-09-10T20:58:53.903833+0000","last_fullsized":"2024-09-10T20:58:53.903833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:39.726666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904184+0000","last_change":"2024-09-10T20:58:46.226623+0000","last_active":"2024-09-10T20:58:53.904184+0000","last_peered":"2024-09-10T20:58:53.904184+0000","last_clean":"2024-09-10T20:58:53.904184+0000","last_became_active":"2024-09-10T20:58:46.226326+0000","last_became_peered":"2024-09-10T20:58:46.226326+0000","last_unstale":"2024-09-10T20:58:53.904184+0000","last_undegraded":"2024-09-10T20:58:53.904184+0000","last_fullsized":"2024-09-10T20:58:53.904184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:02:21.934804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256251+0000","last_change":"2024-09-10T20:58:51.556678+0000","last_active":"2024-09-10T20:58:52.256251+0000","last_peered":"2024-09-10T20:58:52.256251+0000","last_clean":"2024-09-10T20:58:52.256251+0000","last_became_active":"2024-09-10T20:58:51.556271+0000","last_became_peered":"2024-09-10T20:58:51.556271+0000","last_unstale":"2024-09-10T20:58:52.256251+0000","last_undegraded":"2024-09-10T20:58:52.256251+0000","last_fullsized":"2024-09-10T20:58:52.256251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:02.438361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904916+0000","last_change":"2024-09-10T20:58:47.243793+0000","last_active":"2024-09-10T20:58:53.904916+0000","last_peered":"2024-09-10T20:58:53.904916+0000","last_clean":"2024-09-10T20:58:53.904916+0000","last_became_active":"2024-09-10T20:58:47.243537+0000","last_became_peered":"2024-09-10T20:58:47.243537+0000","last_unstale":"2024-09-10T20:58:53.904916+0000","last_undegraded":"2024-09-10T20:58:53.904916+0000","last_fullsized":"2024-09-10T20:58:53.904916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:04.464193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:56.612638+0000","last_change":"2024-09-10T20:58:46.225562+0000","last_active":"2024-09-10T20:58:56.612638+0000","last_peered":"2024-09-10T20:58:56.612638+0000","last_clean":"2024-09-10T20:58:56.612638+0000","last_became_active":"2024-09-10T20:58:46.225260+0000","last_became_peered":"2024-09-10T20:58:46.225260+0000","last_unstale":"2024-09-10T20:58:56.612638+0000","last_undegraded":"2024-09-10T20:58:56.612638+0000","last_fullsized":"2024-09-10T20:58:56.612638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:31.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316695+0000","last_change":"2024-09-10T20:58:51.278993+0000","last_active":"2024-09-10T20:58:53.316695+0000","last_peered":"2024-09-10T20:58:53.316695+0000","last_clean":"2024-09-10T20:58:53.316695+0000","last_became_active":"2024-09-10T20:58:51.278069+0000","last_became_peered":"2024-09-10T20:58:51.278069+0000","last_unstale":"2024-09-10T20:58:53.316695+0000","last_undegraded":"2024-09-10T20:58:53.316695+0000","last_fullsized":"2024-09-10T20:58:53.316695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:21.828620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257045+0000","last_change":"2024-09-10T20:58:47.235717+0000","last_active":"2024-09-10T20:58:52.257045+0000","last_peered":"2024-09-10T20:58:52.257045+0000","last_clean":"2024-09-10T20:58:52.257045+0000","last_became_active":"2024-09-10T20:58:47.235451+0000","last_became_peered":"2024-09-10T20:58:47.235451+0000","last_unstale":"2024-09-10T20:58:52.257045+0000","last_undegraded":"2024-09-10T20:58:52.257045+0000","last_fullsized":"2024-09-10T20:58:52.257045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:28.134150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316771+0000","last_change":"2024-09-10T20:58:46.226035+0000","last_active":"2024-09-10T20:58:53.316771+0000","last_peered":"2024-09-10T20:58:53.316771+0000","last_clean":"2024-09-10T20:58:53.316771+0000","last_became_active":"2024-09-10T20:58:46.225778+0000","last_became_peered":"2024-09-10T20:58:46.225778+0000","last_unstale":"2024-09-10T20:58:53.316771+0000","last_undegraded":"2024-09-10T20:58:53.316771+0000","last_fullsized":"2024-09-10T20:58:53.316771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:57.175951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256534+0000","last_change":"2024-09-10T20:58:51.274974+0000","last_active":"2024-09-10T20:58:52.256534+0000","last_peered":"2024-09-10T20:58:52.256534+0000","last_clean":"2024-09-10T20:58:52.256534+0000","last_became_active":"2024-09-10T20:58:51.274597+0000","last_became_peered":"2024-09-10T20:58:51.274597+0000","last_unstale":"2024-09-10T20:58:52.256534+0000","last_undegraded":"2024-09-10T20:58:52.256534+0000","last_fullsized":"2024-09-10T20:58:52.256534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:31.184777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904058+0000","last_change":"2024-09-10T20:58:47.245724+0000","last_active":"2024-09-10T20:58:53.904058+0000","last_peered":"2024-09-10T20:58:53.904058+0000","last_clean":"2024-09-10T20:58:53.904058+0000","last_became_active":"2024-09-10T20:58:47.245252+0000","last_became_peered":"2024-09-10T20:58:47.245252+0000","last_unstale":"2024-09-10T20:58:53.904058+0000","last_undegraded":"2024-09-10T20:58:53.904058+0000","last_fullsized":"2024-09-10T20:58:53.904058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:29.836234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255635+0000","last_change":"2024-09-10T20:58:46.229938+0000","last_active":"2024-09-10T20:58:52.255635+0000","last_peered":"2024-09-10T20:58:52.255635+0000","last_clean":"2024-09-10T20:58:52.255635+0000","last_became_active":"2024-09-10T20:58:46.229121+0000","last_became_peered":"2024-09-10T20:58:46.229121+0000","last_unstale":"2024-09-10T20:58:52.255635+0000","last_undegraded":"2024-09-10T20:58:52.255635+0000","last_fullsized":"2024-09-10T20:58:52.255635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:18.355281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258591+0000","last_change":"2024-09-10T20:58:51.280436+0000","last_active":"2024-09-10T20:58:52.258591+0000","last_peered":"2024-09-10T20:58:52.258591+0000","last_clean":"2024-09-10T20:58:52.258591+0000","last_became_active":"2024-09-10T20:58:51.279981+0000","last_became_peered":"2024-09-10T20:58:51.279981+0000","last_unstale":"2024-09-10T20:58:52.258591+0000","last_undegraded":"2024-09-10T20:58:52.258591+0000","last_fullsized":"2024-09-10T20:58:52.258591+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:01.843001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255541+0000","last_change":"2024-09-10T20:58:47.235938+0000","last_active":"2024-09-10T20:58:52.255541+0000","last_peered":"2024-09-10T20:58:52.255541+0000","last_clean":"2024-09-10T20:58:52.255541+0000","last_became_active":"2024-09-10T20:58:47.235628+0000","last_became_peered":"2024-09-10T20:58:47.235628+0000","last_unstale":"2024-09-10T20:58:52.255541+0000","last_undegraded":"2024-09-10T20:58:52.255541+0000","last_fullsized":"2024-09-10T20:58:52.255541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:13.462491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904541+0000","last_change":"2024-09-10T20:58:46.230618+0000","last_active":"2024-09-10T20:58:53.904541+0000","last_peered":"2024-09-10T20:58:53.904541+0000","last_clean":"2024-09-10T20:58:53.904541+0000","last_became_active":"2024-09-10T20:58:46.230289+0000","last_became_peered":"2024-09-10T20:58:46.230289+0000","last_unstale":"2024-09-10T20:58:53.904541+0000","last_undegraded":"2024-09-10T20:58:53.904541+0000","last_fullsized":"2024-09-10T20:58:53.904541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:51.217167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258193+0000","last_change":"2024-09-10T20:58:51.557495+0000","last_active":"2024-09-10T20:58:52.258193+0000","last_peered":"2024-09-10T20:58:52.258193+0000","last_clean":"2024-09-10T20:58:52.258193+0000","last_became_active":"2024-09-10T20:58:51.556492+0000","last_became_peered":"2024-09-10T20:58:51.556492+0000","last_unstale":"2024-09-10T20:58:52.258193+0000","last_undegraded":"2024-09-10T20:58:52.258193+0000","last_fullsized":"2024-09-10T20:58:52.258193+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:48.716121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903643+0000","last_change":"2024-09-10T20:58:51.555588+0000","last_active":"2024-09-10T20:58:53.903643+0000","last_peered":"2024-09-10T20:58:53.903643+0000","last_clean":"2024-09-10T20:58:53.903643+0000","last_became_active":"2024-09-10T20:58:51.555232+0000","last_became_peered":"2024-09-10T20:58:51.555232+0000","last_unstale":"2024-09-10T20:58:53.903643+0000","last_undegraded":"2024-09-10T20:58:53.903643+0000","last_fullsized":"2024-09-10T20:58:53.903643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.486482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316315+0000","last_change":"2024-09-10T20:58:47.239189+0000","last_active":"2024-09-10T20:58:53.316315+0000","last_peered":"2024-09-10T20:58:53.316315+0000","last_clean":"2024-09-10T20:58:53.316315+0000","last_became_active":"2024-09-10T20:58:47.238926+0000","last_became_peered":"2024-09-10T20:58:47.238926+0000","last_unstale":"2024-09-10T20:58:53.316315+0000","last_undegraded":"2024-09-10T20:58:53.316315+0000","last_fullsized":"2024-09-10T20:58:53.316315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:02.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255350+0000","last_change":"2024-09-10T20:58:46.229369+0000","last_active":"2024-09-10T20:58:52.255350+0000","last_peered":"2024-09-10T20:58:52.255350+0000","last_clean":"2024-09-10T20:58:52.255350+0000","last_became_active":"2024-09-10T20:58:46.229094+0000","last_became_peered":"2024-09-10T20:58:46.229094+0000","last_unstale":"2024-09-10T20:58:52.255350+0000","last_undegraded":"2024-09-10T20:58:52.255350+0000","last_fullsized":"2024-09-10T20:58:52.255350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:20.646322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904390+0000","last_change":"2024-09-10T20:58:51.280161+0000","last_active":"2024-09-10T20:58:53.904390+0000","last_peered":"2024-09-10T20:58:53.904390+0000","last_clean":"2024-09-10T20:58:53.904390+0000","last_became_active":"2024-09-10T20:58:51.279714+0000","last_became_peered":"2024-09-10T20:58:51.279714+0000","last_unstale":"2024-09-10T20:58:53.904390+0000","last_undegraded":"2024-09-10T20:58:53.904390+0000","last_fullsized":"2024-09-10T20:58:53.904390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.486482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258577+0000","last_change":"2024-09-10T20:58:47.248033+0000","last_active":"2024-09-10T20:58:52.258577+0000","last_peered":"2024-09-10T20:58:52.258577+0000","last_clean":"2024-09-10T20:58:52.258577+0000","last_became_active":"2024-09-10T20:58:47.247761+0000","last_became_peered":"2024-09-10T20:58:47.247761+0000","last_unstale":"2024-09-10T20:58:52.258577+0000","last_undegraded":"2024-09-10T20:58:52.258577+0000","last_fullsized":"2024-09-10T20:58:52.258577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:04.498018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258357+0000","last_change":"2024-09-10T20:58:46.223392+0000","last_active":"2024-09-10T20:58:52.258357+0000","last_peered":"2024-09-10T20:58:52.258357+0000","last_clean":"2024-09-10T20:58:52.258357+0000","last_became_active":"2024-09-10T20:58:46.223123+0000","last_became_peered":"2024-09-10T20:58:46.223123+0000","last_unstale":"2024-09-10T20:58:52.258357+0000","last_undegraded":"2024-09-10T20:58:52.258357+0000","last_fullsized":"2024-09-10T20:58:52.258357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:16.570128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904433+0000","last_change":"2024-09-10T20:58:46.227651+0000","last_active":"2024-09-10T20:58:53.904433+0000","last_peered":"2024-09-10T20:58:53.904433+0000","last_clean":"2024-09-10T20:58:53.904433+0000","last_became_active":"2024-09-10T20:58:46.226418+0000","last_became_peered":"2024-09-10T20:58:46.226418+0000","last_unstale":"2024-09-10T20:58:53.904433+0000","last_undegraded":"2024-09-10T20:58:53.904433+0000","last_fullsized":"2024-09-10T20:58:53.904433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:02.951532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904158+0000","last_change":"2024-09-10T20:58:47.245394+0000","last_active":"2024-09-10T20:58:53.904158+0000","last_peered":"2024-09-10T20:58:53.904158+0000","last_clean":"2024-09-10T20:58:53.904158+0000","last_became_active":"2024-09-10T20:58:47.243973+0000","last_became_peered":"2024-09-10T20:58:47.243973+0000","last_unstale":"2024-09-10T20:58:53.904158+0000","last_undegraded":"2024-09-10T20:58:53.904158+0000","last_fullsized":"2024-09-10T20:58:53.904158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:54.993134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903643+0000","last_change":"2024-09-10T20:58:51.274539+0000","last_active":"2024-09-10T20:58:53.903643+0000","last_peered":"2024-09-10T20:58:53.903643+0000","last_clean":"2024-09-10T20:58:53.903643+0000","last_became_active":"2024-09-10T20:58:51.274124+0000","last_became_peered":"2024-09-10T20:58:51.274124+0000","last_unstale":"2024-09-10T20:58:53.903643+0000","last_undegraded":"2024-09-10T20:58:53.903643+0000","last_fullsized":"2024-09-10T20:58:53.903643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:20.469415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258431+0000","last_change":"2024-09-10T20:58:46.223353+0000","last_active":"2024-09-10T20:58:52.258431+0000","last_peered":"2024-09-10T20:58:52.258431+0000","last_clean":"2024-09-10T20:58:52.258431+0000","last_became_active":"2024-09-10T20:58:46.223059+0000","last_became_peered":"2024-09-10T20:58:46.223059+0000","last_unstale":"2024-09-10T20:58:52.258431+0000","last_undegraded":"2024-09-10T20:58:52.258431+0000","last_fullsized":"2024-09-10T20:58:52.258431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:02:21.934804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257112+0000","last_change":"2024-09-10T20:58:47.233445+0000","last_active":"2024-09-10T20:58:52.257112+0000","last_peered":"2024-09-10T20:58:52.257112+0000","last_clean":"2024-09-10T20:58:52.257112+0000","last_became_active":"2024-09-10T20:58:47.233157+0000","last_became_peered":"2024-09-10T20:58:47.233157+0000","last_unstale":"2024-09-10T20:58:52.257112+0000","last_undegraded":"2024-09-10T20:58:52.257112+0000","last_fullsized":"2024-09-10T20:58:52.257112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:52.222165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258673+0000","last_change":"2024-09-10T20:58:51.281498+0000","last_active":"2024-09-10T20:58:52.258673+0000","last_peered":"2024-09-10T20:58:52.258673+0000","last_clean":"2024-09-10T20:58:52.258673+0000","last_became_active":"2024-09-10T20:58:51.281027+0000","last_became_peered":"2024-09-10T20:58:51.281027+0000","last_unstale":"2024-09-10T20:58:52.258673+0000","last_undegraded":"2024-09-10T20:58:52.258673+0000","last_fullsized":"2024-09-10T20:58:52.258673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:35.279483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903782+0000","last_change":"2024-09-10T20:58:47.247788+0000","last_active":"2024-09-10T20:58:53.903782+0000","last_peered":"2024-09-10T20:58:53.903782+0000","last_clean":"2024-09-10T20:58:53.903782+0000","last_became_active":"2024-09-10T20:58:47.247513+0000","last_became_peered":"2024-09-10T20:58:47.247513+0000","last_unstale":"2024-09-10T20:58:53.903782+0000","last_undegraded":"2024-09-10T20:58:53.903782+0000","last_fullsized":"2024-09-10T20:58:53.903782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:59.707834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903324+0000","last_change":"2024-09-10T20:58:47.232038+0000","last_active":"2024-09-10T20:58:53.903324+0000","last_peered":"2024-09-10T20:58:53.903324+0000","last_clean":"2024-09-10T20:58:53.903324+0000","last_became_active":"2024-09-10T20:58:47.231590+0000","last_became_peered":"2024-09-10T20:58:47.231590+0000","last_unstale":"2024-09-10T20:58:53.903324+0000","last_undegraded":"2024-09-10T20:58:53.903324+0000","last_fullsized":"2024-09-10T20:58:53.903324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:26.466044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255659+0000","last_change":"2024-09-10T20:58:47.237625+0000","last_active":"2024-09-10T20:58:52.255659+0000","last_peered":"2024-09-10T20:58:52.255659+0000","last_clean":"2024-09-10T20:58:52.255659+0000","last_became_active":"2024-09-10T20:58:47.237247+0000","last_became_peered":"2024-09-10T20:58:47.237247+0000","last_unstale":"2024-09-10T20:58:52.255659+0000","last_undegraded":"2024-09-10T20:58:52.255659+0000","last_fullsized":"2024-09-10T20:58:52.255659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:21.642999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904061+0000","last_change":"2024-09-10T20:58:47.247495+0000","last_active":"2024-09-10T20:58:53.904061+0000","last_peered":"2024-09-10T20:58:53.904061+0000","last_clean":"2024-09-10T20:58:53.904061+0000","last_became_active":"2024-09-10T20:58:47.247049+0000","last_became_peered":"2024-09-10T20:58:47.247049+0000","last_unstale":"2024-09-10T20:58:53.904061+0000","last_undegraded":"2024-09-10T20:58:53.904061+0000","last_fullsized":"2024-09-10T20:58:53.904061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:18.950297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258717+0000","last_change":"2024-09-10T20:58:47.243173+0000","last_active":"2024-09-10T20:58:52.258717+0000","last_peered":"2024-09-10T20:58:52.258717+0000","last_clean":"2024-09-10T20:58:52.258717+0000","last_became_active":"2024-09-10T20:58:47.242521+0000","last_became_peered":"2024-09-10T20:58:47.242521+0000","last_unstale":"2024-09-10T20:58:52.258717+0000","last_undegraded":"2024-09-10T20:58:52.258717+0000","last_fullsized":"2024-09-10T20:58:52.258717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:54.993134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316600+0000","last_change":"2024-09-10T20:58:47.238462+0000","last_active":"2024-09-10T20:58:53.316600+0000","last_peered":"2024-09-10T20:58:53.316600+0000","last_clean":"2024-09-10T20:58:53.316600+0000","last_became_active":"2024-09-10T20:58:47.238166+0000","last_became_peered":"2024-09-10T20:58:47.238166+0000","last_unstale":"2024-09-10T20:58:53.316600+0000","last_undegraded":"2024-09-10T20:58:53.316600+0000","last_fullsized":"2024-09-10T20:58:53.316600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:51.719957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904758+0000","last_change":"2024-09-10T20:58:47.245403+0000","last_active":"2024-09-10T20:58:53.904758+0000","last_peered":"2024-09-10T20:58:53.904758+0000","last_clean":"2024-09-10T20:58:53.904758+0000","last_became_active":"2024-09-10T20:58:47.245125+0000","last_became_peered":"2024-09-10T20:58:47.245125+0000","last_unstale":"2024-09-10T20:58:53.904758+0000","last_undegraded":"2024-09-10T20:58:53.904758+0000","last_fullsized":"2024-09-10T20:58:53.904758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:29.836234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.259179+0000","last_change":"2024-09-10T20:58:47.242826+0000","last_active":"2024-09-10T20:58:52.259179+0000","last_peered":"2024-09-10T20:58:52.259179+0000","last_clean":"2024-09-10T20:58:52.259179+0000","last_became_active":"2024-09-10T20:58:47.242462+0000","last_became_peered":"2024-09-10T20:58:47.242462+0000","last_unstale":"2024-09-10T20:58:52.259179+0000","last_undegraded":"2024-09-10T20:58:52.259179+0000","last_fullsized":"2024-09-10T20:58:52.259179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:33.221761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255216+0000","last_change":"2024-09-10T20:58:47.236478+0000","last_active":"2024-09-10T20:58:52.255216+0000","last_peered":"2024-09-10T20:58:52.255216+0000","last_clean":"2024-09-10T20:58:52.255216+0000","last_became_active":"2024-09-10T20:58:47.236141+0000","last_became_peered":"2024-09-10T20:58:47.236141+0000","last_unstale":"2024-09-10T20:58:52.255216+0000","last_undegraded":"2024-09-10T20:58:52.255216+0000","last_fullsized":"2024-09-10T20:58:52.255216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:51.719957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258257+0000","last_change":"2024-09-10T20:58:47.233386+0000","last_active":"2024-09-10T20:58:52.258257+0000","last_peered":"2024-09-10T20:58:52.258257+0000","last_clean":"2024-09-10T20:58:52.258257+0000","last_became_active":"2024-09-10T20:58:47.233110+0000","last_became_peered":"2024-09-10T20:58:47.233110+0000","last_unstale":"2024-09-10T20:58:52.258257+0000","last_undegraded":"2024-09-10T20:58:52.258257+0000","last_fullsized":"2024-09-10T20:58:52.258257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:18.950297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255532+0000","last_change":"2024-09-10T20:58:47.236278+0000","last_active":"2024-09-10T20:58:52.255532+0000","last_peered":"2024-09-10T20:58:52.255532+0000","last_clean":"2024-09-10T20:58:52.255532+0000","last_became_active":"2024-09-10T20:58:47.236005+0000","last_became_peered":"2024-09-10T20:58:47.236005+0000","last_unstale":"2024-09-10T20:58:52.255532+0000","last_undegraded":"2024-09-10T20:58:52.255532+0000","last_fullsized":"2024-09-10T20:58:52.255532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:34:54.893698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903878+0000","last_change":"2024-09-10T20:58:47.247778+0000","last_active":"2024-09-10T20:58:53.903878+0000","last_peered":"2024-09-10T20:58:53.903878+0000","last_clean":"2024-09-10T20:58:53.903878+0000","last_became_active":"2024-09-10T20:58:47.247103+0000","last_became_peered":"2024-09-10T20:58:47.247103+0000","last_unstale":"2024-09-10T20:58:53.903878+0000","last_undegraded":"2024-09-10T20:58:53.903878+0000","last_fullsized":"2024-09-10T20:58:53.903878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:42.546277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904334+0000","last_change":"2024-09-10T20:58:47.230471+0000","last_active":"2024-09-10T20:58:53.904334+0000","last_peered":"2024-09-10T20:58:53.904334+0000","last_clean":"2024-09-10T20:58:53.904334+0000","last_became_active":"2024-09-10T20:58:47.230174+0000","last_became_peered":"2024-09-10T20:58:47.230174+0000","last_unstale":"2024-09-10T20:58:53.904334+0000","last_undegraded":"2024-09-10T20:58:53.904334+0000","last_fullsized":"2024-09-10T20:58:53.904334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:54.301892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905155+0000","last_change":"2024-09-10T20:58:47.245889+0000","last_active":"2024-09-10T20:58:53.905155+0000","last_peered":"2024-09-10T20:58:53.905155+0000","last_clean":"2024-09-10T20:58:53.905155+0000","last_became_active":"2024-09-10T20:58:47.245547+0000","last_became_peered":"2024-09-10T20:58:47.245547+0000","last_unstale":"2024-09-10T20:58:53.905155+0000","last_undegraded":"2024-09-10T20:58:53.905155+0000","last_fullsized":"2024-09-10T20:58:53.905155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:11.073447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903659+0000","last_change":"2024-09-10T20:58:47.234594+0000","last_active":"2024-09-10T20:58:53.903659+0000","last_peered":"2024-09-10T20:58:53.903659+0000","last_clean":"2024-09-10T20:58:53.903659+0000","last_became_active":"2024-09-10T20:58:47.234275+0000","last_became_peered":"2024-09-10T20:58:47.234275+0000","last_unstale":"2024-09-10T20:58:53.903659+0000","last_undegraded":"2024-09-10T20:58:53.903659+0000","last_fullsized":"2024-09-10T20:58:53.903659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:25.490881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256889+0000","last_change":"2024-09-10T20:58:47.236653+0000","last_active":"2024-09-10T20:58:52.256889+0000","last_peered":"2024-09-10T20:58:52.256889+0000","last_clean":"2024-09-10T20:58:52.256889+0000","last_became_active":"2024-09-10T20:58:47.236361+0000","last_became_peered":"2024-09-10T20:58:47.236361+0000","last_unstale":"2024-09-10T20:58:52.256889+0000","last_undegraded":"2024-09-10T20:58:52.256889+0000","last_fullsized":"2024-09-10T20:58:52.256889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:50.078953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255103+0000","last_change":"2024-09-10T20:58:47.247779+0000","last_active":"2024-09-10T20:58:52.255103+0000","last_peered":"2024-09-10T20:58:52.255103+0000","last_clean":"2024-09-10T20:58:52.255103+0000","last_became_active":"2024-09-10T20:58:47.246921+0000","last_became_peered":"2024-09-10T20:58:47.246921+0000","last_unstale":"2024-09-10T20:58:52.255103+0000","last_undegraded":"2024-09-10T20:58:52.255103+0000","last_fullsized":"2024-09-10T20:58:52.255103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:07.109711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316370+0000","last_change":"2024-09-10T20:58:47.233169+0000","last_active":"2024-09-10T20:58:53.316370+0000","last_peered":"2024-09-10T20:58:53.316370+0000","last_clean":"2024-09-10T20:58:53.316370+0000","last_became_active":"2024-09-10T20:58:47.232869+0000","last_became_peered":"2024-09-10T20:58:47.232869+0000","last_unstale":"2024-09-10T20:58:53.316370+0000","last_undegraded":"2024-09-10T20:58:53.316370+0000","last_fullsized":"2024-09-10T20:58:53.316370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:37.216505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903723+0000","last_change":"2024-09-10T20:58:47.241778+0000","last_active":"2024-09-10T20:58:53.903723+0000","last_peered":"2024-09-10T20:58:53.903723+0000","last_clean":"2024-09-10T20:58:53.903723+0000","last_became_active":"2024-09-10T20:58:47.241164+0000","last_became_peered":"2024-09-10T20:58:47.241164+0000","last_unstale":"2024-09-10T20:58:53.903723+0000","last_undegraded":"2024-09-10T20:58:53.903723+0000","last_fullsized":"2024-09-10T20:58:53.903723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:59.707834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258272+0000","last_change":"2024-09-10T20:58:47.234831+0000","last_active":"2024-09-10T20:58:52.258272+0000","last_peered":"2024-09-10T20:58:52.258272+0000","last_clean":"2024-09-10T20:58:52.258272+0000","last_became_active":"2024-09-10T20:58:47.234566+0000","last_became_peered":"2024-09-10T20:58:47.234566+0000","last_unstale":"2024-09-10T20:58:52.258272+0000","last_undegraded":"2024-09-10T20:58:52.258272+0000","last_fullsized":"2024-09-10T20:58:52.258272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:29.836234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257006+0000","last_change":"2024-09-10T20:58:47.231297+0000","last_active":"2024-09-10T20:58:52.257006+0000","last_peered":"2024-09-10T20:58:52.257006+0000","last_clean":"2024-09-10T20:58:52.257006+0000","last_became_active":"2024-09-10T20:58:47.230943+0000","last_became_peered":"2024-09-10T20:58:47.230943+0000","last_unstale":"2024-09-10T20:58:52.257006+0000","last_undegraded":"2024-09-10T20:58:52.257006+0000","last_fullsized":"2024-09-10T20:58:52.257006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:29.113846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904210+0000","last_change":"2024-09-10T20:58:47.232404+0000","last_active":"2024-09-10T20:58:53.904210+0000","last_peered":"2024-09-10T20:58:53.904210+0000","last_clean":"2024-09-10T20:58:53.904210+0000","last_became_active":"2024-09-10T20:58:47.232127+0000","last_became_peered":"2024-09-10T20:58:47.232127+0000","last_unstale":"2024-09-10T20:58:53.904210+0000","last_undegraded":"2024-09-10T20:58:53.904210+0000","last_fullsized":"2024-09-10T20:58:53.904210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:39.726666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904756+0000","last_change":"2024-09-10T20:58:47.231816+0000","last_active":"2024-09-10T20:58:53.904756+0000","last_peered":"2024-09-10T20:58:53.904756+0000","last_clean":"2024-09-10T20:58:53.904756+0000","last_became_active":"2024-09-10T20:58:47.231545+0000","last_became_peered":"2024-09-10T20:58:47.231545+0000","last_unstale":"2024-09-10T20:58:53.904756+0000","last_undegraded":"2024-09-10T20:58:53.904756+0000","last_fullsized":"2024-09-10T20:58:53.904756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:54.993134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905112+0000","last_change":"2024-09-10T20:58:47.246656+0000","last_active":"2024-09-10T20:58:53.905112+0000","last_peered":"2024-09-10T20:58:53.905112+0000","last_clean":"2024-09-10T20:58:53.905112+0000","last_became_active":"2024-09-10T20:58:47.245647+0000","last_became_peered":"2024-09-10T20:58:47.245647+0000","last_unstale":"2024-09-10T20:58:53.905112+0000","last_undegraded":"2024-09-10T20:58:53.905112+0000","last_fullsized":"2024-09-10T20:58:53.905112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:35.158790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258006+0000","last_change":"2024-09-10T20:58:47.235102+0000","last_active":"2024-09-10T20:58:52.258006+0000","last_peered":"2024-09-10T20:58:52.258006+0000","last_clean":"2024-09-10T20:58:52.258006+0000","last_became_active":"2024-09-10T20:58:47.233909+0000","last_became_peered":"2024-09-10T20:58:47.233909+0000","last_unstale":"2024-09-10T20:58:52.258006+0000","last_undegraded":"2024-09-10T20:58:52.258006+0000","last_fullsized":"2024-09-10T20:58:52.258006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:59.707834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904152+0000","last_change":"2024-09-10T20:58:47.242793+0000","last_active":"2024-09-10T20:58:53.904152+0000","last_peered":"2024-09-10T20:58:53.904152+0000","last_clean":"2024-09-10T20:58:53.904152+0000","last_became_active":"2024-09-10T20:58:47.242495+0000","last_became_peered":"2024-09-10T20:58:47.242495+0000","last_unstale":"2024-09-10T20:58:53.904152+0000","last_undegraded":"2024-09-10T20:58:53.904152+0000","last_fullsized":"2024-09-10T20:58:53.904152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:21.457962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255428+0000","last_change":"2024-09-10T20:58:47.244668+0000","last_active":"2024-09-10T20:58:52.255428+0000","last_peered":"2024-09-10T20:58:52.255428+0000","last_clean":"2024-09-10T20:58:52.255428+0000","last_became_active":"2024-09-10T20:58:47.244110+0000","last_became_peered":"2024-09-10T20:58:47.244110+0000","last_unstale":"2024-09-10T20:58:52.255428+0000","last_undegraded":"2024-09-10T20:58:52.255428+0000","last_fullsized":"2024-09-10T20:58:52.255428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:41.633550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904570+0000","last_change":"2024-09-10T20:58:47.241409+0000","last_active":"2024-09-10T20:58:53.904570+0000","last_peered":"2024-09-10T20:58:53.904570+0000","last_clean":"2024-09-10T20:58:53.904570+0000","last_became_active":"2024-09-10T20:58:47.241104+0000","last_became_peered":"2024-09-10T20:58:47.241104+0000","last_unstale":"2024-09-10T20:58:53.904570+0000","last_undegraded":"2024-09-10T20:58:53.904570+0000","last_fullsized":"2024-09-10T20:58:53.904570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:08.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255561+0000","last_change":"2024-09-10T20:58:47.246735+0000","last_active":"2024-09-10T20:58:52.255561+0000","last_peered":"2024-09-10T20:58:52.255561+0000","last_clean":"2024-09-10T20:58:52.255561+0000","last_became_active":"2024-09-10T20:58:47.246491+0000","last_became_peered":"2024-09-10T20:58:47.246491+0000","last_unstale":"2024-09-10T20:58:52.255561+0000","last_undegraded":"2024-09-10T20:58:52.255561+0000","last_fullsized":"2024-09-10T20:58:52.255561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:16.591593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904106+0000","last_change":"2024-09-10T20:58:47.246507+0000","last_active":"2024-09-10T20:58:53.904106+0000","last_peered":"2024-09-10T20:58:53.904106+0000","last_clean":"2024-09-10T20:58:53.904106+0000","last_became_active":"2024-09-10T20:58:47.245963+0000","last_became_peered":"2024-09-10T20:58:47.245963+0000","last_unstale":"2024-09-10T20:58:53.904106+0000","last_undegraded":"2024-09-10T20:58:53.904106+0000","last_fullsized":"2024-09-10T20:58:53.904106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:04.498018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316869+0000","last_change":"2024-09-10T20:58:47.242014+0000","last_active":"2024-09-10T20:58:53.316869+0000","last_peered":"2024-09-10T20:58:53.316869+0000","last_clean":"2024-09-10T20:58:53.316869+0000","last_became_active":"2024-09-10T20:58:47.241727+0000","last_became_peered":"2024-09-10T20:58:47.241727+0000","last_unstale":"2024-09-10T20:58:53.316869+0000","last_undegraded":"2024-09-10T20:58:53.316869+0000","last_fullsized":"2024-09-10T20:58:53.316869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:13.462491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904966+0000","last_change":"2024-09-10T20:58:47.246360+0000","last_active":"2024-09-10T20:58:53.904966+0000","last_peered":"2024-09-10T20:58:53.904966+0000","last_clean":"2024-09-10T20:58:53.904966+0000","last_became_active":"2024-09-10T20:58:47.245363+0000","last_became_peered":"2024-09-10T20:58:47.245363+0000","last_unstale":"2024-09-10T20:58:53.904966+0000","last_undegraded":"2024-09-10T20:58:53.904966+0000","last_fullsized":"2024-09-10T20:58:53.904966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:59.710882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":693653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":102829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":102863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":693709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":262144,"data_stored":102793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:59:07.262 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:59:07.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.389+0000 7f3ffb447640 1 -- 172.21.15.98:0/799833564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff40b0d50 msgr2=0x7f3ff40af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.389+0000 7f3ffb447640 1 --2- 172.21.15.98:0/799833564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff40b0d50 0x7f3ff40af5b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009830 tx=0x7f3fe802a480 comp rx=0 tx=0).stop 2024-09-10T20:59:07.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.389+0000 7f3ffb447640 1 -- 172.21.15.98:0/799833564 shutdown_connections 2024-09-10T20:59:07.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.389+0000 7f3ffb447640 1 --2- 172.21.15.98:0/799833564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff40b0d50 0x7f3ff40af5b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.389+0000 7f3ffb447640 1 -- 172.21.15.98:0/799833564 >> 172.21.15.98:0/799833564 conn(0x7f3ff40b37c0 msgr2=0x7f3ff40b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:07.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.390+0000 7f3ffb447640 1 -- 172.21.15.98:0/799833564 shutdown_connections 2024-09-10T20:59:07.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.390+0000 7f3ffb447640 1 -- 172.21.15.98:0/799833564 wait complete. 2024-09-10T20:59:07.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.391+0000 7f3ffb447640 1 Processor -- start 2024-09-10T20:59:07.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.391+0000 7f3ffb447640 1 -- start start 2024-09-10T20:59:07.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.391+0000 7f3ffb447640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ff40cff70 0x7f3ff4153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:07.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.391+0000 7f3ffb447640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ff4160530 0x7f3ff41f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:07.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.391+0000 7f3ffb447640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff40ca930 0x7f3ff40cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:07.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.391+0000 7f3ffb447640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff40b0200 con 0x7f3ff40ca930 2024-09-10T20:59:07.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.391+0000 7f3ffb447640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff40b34f0 con 0x7f3ff40cff70 2024-09-10T20:59:07.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.391+0000 7f3ffb447640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff40b0380 con 0x7f3ff4160530 2024-09-10T20:59:07.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ff99e4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff40ca930 0x7f3ff40cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:07.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ff99e4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff40ca930 0x7f3ff40cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58456/0 (socket says 172.21.15.98:58456) 2024-09-10T20:59:07.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ff99e4640 1 -- 172.21.15.98:0/2996212136 learned_addr learned my addr 172.21.15.98:0/2996212136 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:07.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ff91e3640 1 --2- 172.21.15.98:0/2996212136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ff40cff70 0x7f3ff4153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:07.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ff89e2640 1 --2- 172.21.15.98:0/2996212136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ff4160530 0x7f3ff41f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:07.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ff99e4640 1 -- 172.21.15.98:0/2996212136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ff4160530 msgr2=0x7f3ff41f6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ff99e4640 1 --2- 172.21.15.98:0/2996212136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ff4160530 0x7f3ff41f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ff99e4640 1 -- 172.21.15.98:0/2996212136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ff40cff70 msgr2=0x7f3ff4153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ff99e4640 1 --2- 172.21.15.98:0/2996212136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ff40cff70 0x7f3ff4153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ff99e4640 1 -- 172.21.15.98:0/2996212136 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff40ca6e0 con 0x7f3ff40ca930 2024-09-10T20:59:07.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ff99e4640 1 --2- 172.21.15.98:0/2996212136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff40ca930 0x7f3ff40cdd70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c7b0 tx=0x7f3fe400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:07.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ff1ffb640 1 -- 172.21.15.98:0/2996212136 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4004b60 con 0x7f3ff40ca930 2024-09-10T20:59:07.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ff1ffb640 1 -- 172.21.15.98:0/2996212136 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fe4004df0 con 0x7f3ff40ca930 2024-09-10T20:59:07.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ffb447640 1 -- 172.21.15.98:0/2996212136 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff40caeb0 con 0x7f3ff40ca930 2024-09-10T20:59:07.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ff1ffb640 1 -- 172.21.15.98:0/2996212136 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe400aa90 con 0x7f3ff40ca930 2024-09-10T20:59:07.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.392+0000 7f3ffb447640 1 -- 172.21.15.98:0/2996212136 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff40cb470 con 0x7f3ff40ca930 2024-09-10T20:59:07.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.393+0000 7f3ffb447640 1 -- 172.21.15.98:0/2996212136 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff41538a0 con 0x7f3ff40ca930 2024-09-10T20:59:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.394+0000 7f3ff1ffb640 1 -- 172.21.15.98:0/2996212136 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3fe400ac30 con 0x7f3ff40ca930 2024-09-10T20:59:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.395+0000 7f3ff1ffb640 1 --2- 172.21.15.98:0/2996212136 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3fe0077f70 0x7f3fe007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.395+0000 7f3ff1ffb640 1 -- 172.21.15.98:0/2996212136 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3fe409a340 con 0x7f3ff40ca930 2024-09-10T20:59:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.395+0000 7f3ff91e3640 1 --2- 172.21.15.98:0/2996212136 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3fe0077f70 0x7f3fe007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.395+0000 7f3ff91e3640 1 --2- 172.21.15.98:0/2996212136 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3fe0077f70 0x7f3fe007a430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3fec007fd0 tx=0x7f3fec009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:07.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.399+0000 7f3ff1ffb640 1 -- 172.21.15.98:0/2996212136 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3fe4020070 con 0x7f3ff40ca930 2024-09-10T20:59:07.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.570+0000 7f3ffb447640 1 -- 172.21.15.98:0/2996212136 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3ff40b0840 con 0x7f3fe0077f70 2024-09-10T20:59:07.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.576+0000 7f3ff1ffb640 1 -- 172.21.15.98:0/2996212136 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f3ff40b0840 con 0x7f3fe0077f70 2024-09-10T20:59:07.583 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:07.585 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-10T20:59:07.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.580+0000 7f3ffb447640 1 -- 172.21.15.98:0/2996212136 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3fe0077f70 msgr2=0x7f3fe007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.580+0000 7f3ffb447640 1 --2- 172.21.15.98:0/2996212136 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3fe0077f70 0x7f3fe007a430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3fec007fd0 tx=0x7f3fec009040 comp rx=0 tx=0).stop 2024-09-10T20:59:07.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.580+0000 7f3ffb447640 1 -- 172.21.15.98:0/2996212136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff40ca930 msgr2=0x7f3ff40cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.580+0000 7f3ffb447640 1 --2- 172.21.15.98:0/2996212136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff40ca930 0x7f3ff40cdd70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c7b0 tx=0x7f3fe400cc70 comp rx=0 tx=0).stop 2024-09-10T20:59:07.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.580+0000 7f3ffb447640 1 -- 172.21.15.98:0/2996212136 shutdown_connections 2024-09-10T20:59:07.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.580+0000 7f3ffb447640 1 --2- 172.21.15.98:0/2996212136 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3fe0077f70 0x7f3fe007a430 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.580+0000 7f3ffb447640 1 --2- 172.21.15.98:0/2996212136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ff40ca930 0x7f3ff40cdd70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.580+0000 7f3ffb447640 1 --2- 172.21.15.98:0/2996212136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ff4160530 0x7f3ff41f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.580+0000 7f3ffb447640 1 --2- 172.21.15.98:0/2996212136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ff40cff70 0x7f3ff4153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.580+0000 7f3ffb447640 1 -- 172.21.15.98:0/2996212136 >> 172.21.15.98:0/2996212136 conn(0x7f3ff40b37c0 msgr2=0x7f3ff40af590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:07.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.580+0000 7f3ffb447640 1 -- 172.21.15.98:0/2996212136 shutdown_connections 2024-09-10T20:59:07.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.580+0000 7f3ffb447640 1 -- 172.21.15.98:0/2996212136 wait complete. 2024-09-10T20:59:07.594 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-10T20:59:06.911648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3317760,"data_stored":2011640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003466"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904144+0000","last_change":"2024-09-10T20:58:46.222874+0000","last_active":"2024-09-10T20:58:53.904144+0000","last_peered":"2024-09-10T20:58:53.904144+0000","last_clean":"2024-09-10T20:58:53.904144+0000","last_became_active":"2024-09-10T20:58:46.222585+0000","last_became_peered":"2024-09-10T20:58:46.222585+0000","last_unstale":"2024-09-10T20:58:53.904144+0000","last_undegraded":"2024-09-10T20:58:53.904144+0000","last_fullsized":"2024-09-10T20:58:53.904144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:12.557183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258747+0000","last_change":"2024-09-10T20:58:47.236553+0000","last_active":"2024-09-10T20:58:52.258747+0000","last_peered":"2024-09-10T20:58:52.258747+0000","last_clean":"2024-09-10T20:58:52.258747+0000","last_became_active":"2024-09-10T20:58:47.236267+0000","last_became_peered":"2024-09-10T20:58:47.236267+0000","last_unstale":"2024-09-10T20:58:52.258747+0000","last_undegraded":"2024-09-10T20:58:52.258747+0000","last_fullsized":"2024-09-10T20:58:52.258747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:54.301892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258094+0000","last_change":"2024-09-10T20:58:51.557088+0000","last_active":"2024-09-10T20:58:52.258094+0000","last_peered":"2024-09-10T20:58:52.258094+0000","last_clean":"2024-09-10T20:58:52.258094+0000","last_became_active":"2024-09-10T20:58:51.556571+0000","last_became_peered":"2024-09-10T20:58:51.556571+0000","last_unstale":"2024-09-10T20:58:52.258094+0000","last_undegraded":"2024-09-10T20:58:52.258094+0000","last_fullsized":"2024-09-10T20:58:52.258094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:32.723152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258861+0000","last_change":"2024-09-10T20:58:46.221311+0000","last_active":"2024-09-10T20:58:52.258861+0000","last_peered":"2024-09-10T20:58:52.258861+0000","last_clean":"2024-09-10T20:58:52.258861+0000","last_became_active":"2024-09-10T20:58:46.221053+0000","last_became_peered":"2024-09-10T20:58:46.221053+0000","last_unstale":"2024-09-10T20:58:52.258861+0000","last_undegraded":"2024-09-10T20:58:52.258861+0000","last_fullsized":"2024-09-10T20:58:52.258861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:02.951532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903969+0000","last_change":"2024-09-10T20:58:47.246604+0000","last_active":"2024-09-10T20:58:53.903969+0000","last_peered":"2024-09-10T20:58:53.903969+0000","last_clean":"2024-09-10T20:58:53.903969+0000","last_became_active":"2024-09-10T20:58:47.244966+0000","last_became_peered":"2024-09-10T20:58:47.244966+0000","last_unstale":"2024-09-10T20:58:53.903969+0000","last_undegraded":"2024-09-10T20:58:53.903969+0000","last_fullsized":"2024-09-10T20:58:53.903969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:54.301892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255041+0000","last_change":"2024-09-10T20:58:51.275679+0000","last_active":"2024-09-10T20:58:52.255041+0000","last_peered":"2024-09-10T20:58:52.255041+0000","last_clean":"2024-09-10T20:58:52.255041+0000","last_became_active":"2024-09-10T20:58:51.275227+0000","last_became_peered":"2024-09-10T20:58:51.275227+0000","last_unstale":"2024-09-10T20:58:52.255041+0000","last_undegraded":"2024-09-10T20:58:52.255041+0000","last_fullsized":"2024-09-10T20:58:52.255041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:07.240517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905109+0000","last_change":"2024-09-10T20:58:46.223303+0000","last_active":"2024-09-10T20:58:53.905109+0000","last_peered":"2024-09-10T20:58:53.905109+0000","last_clean":"2024-09-10T20:58:53.905109+0000","last_became_active":"2024-09-10T20:58:46.222924+0000","last_became_peered":"2024-09-10T20:58:46.222924+0000","last_unstale":"2024-09-10T20:58:53.905109+0000","last_undegraded":"2024-09-10T20:58:53.905109+0000","last_fullsized":"2024-09-10T20:58:53.905109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:37.987408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255679+0000","last_change":"2024-09-10T20:58:47.231063+0000","last_active":"2024-09-10T20:58:52.255679+0000","last_peered":"2024-09-10T20:58:52.255679+0000","last_clean":"2024-09-10T20:58:52.255679+0000","last_became_active":"2024-09-10T20:58:47.230764+0000","last_became_peered":"2024-09-10T20:58:47.230764+0000","last_unstale":"2024-09-10T20:58:52.255679+0000","last_undegraded":"2024-09-10T20:58:52.255679+0000","last_fullsized":"2024-09-10T20:58:52.255679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:17.763154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256830+0000","last_change":"2024-09-10T20:58:51.275019+0000","last_active":"2024-09-10T20:58:52.256830+0000","last_peered":"2024-09-10T20:58:52.256830+0000","last_clean":"2024-09-10T20:58:52.256830+0000","last_became_active":"2024-09-10T20:58:51.274298+0000","last_became_peered":"2024-09-10T20:58:51.274298+0000","last_unstale":"2024-09-10T20:58:52.256830+0000","last_undegraded":"2024-09-10T20:58:52.256830+0000","last_fullsized":"2024-09-10T20:58:52.256830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.486482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904772+0000","last_change":"2024-09-10T20:58:46.231109+0000","last_active":"2024-09-10T20:58:53.904772+0000","last_peered":"2024-09-10T20:58:53.904772+0000","last_clean":"2024-09-10T20:58:53.904772+0000","last_became_active":"2024-09-10T20:58:46.230798+0000","last_became_peered":"2024-09-10T20:58:46.230798+0000","last_unstale":"2024-09-10T20:58:53.904772+0000","last_undegraded":"2024-09-10T20:58:53.904772+0000","last_fullsized":"2024-09-10T20:58:53.904772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:31.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256327+0000","last_change":"2024-09-10T20:58:47.230225+0000","last_active":"2024-09-10T20:58:52.256327+0000","last_peered":"2024-09-10T20:58:52.256327+0000","last_clean":"2024-09-10T20:58:52.256327+0000","last_became_active":"2024-09-10T20:58:47.229943+0000","last_became_peered":"2024-09-10T20:58:47.229943+0000","last_unstale":"2024-09-10T20:58:52.256327+0000","last_undegraded":"2024-09-10T20:58:52.256327+0000","last_fullsized":"2024-09-10T20:58:52.256327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:45.849686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256190+0000","last_change":"2024-09-10T20:58:51.270577+0000","last_active":"2024-09-10T20:58:52.256190+0000","last_peered":"2024-09-10T20:58:52.256190+0000","last_clean":"2024-09-10T20:58:52.256190+0000","last_became_active":"2024-09-10T20:58:51.270223+0000","last_became_peered":"2024-09-10T20:58:51.270223+0000","last_unstale":"2024-09-10T20:58:52.256190+0000","last_undegraded":"2024-09-10T20:58:52.256190+0000","last_fullsized":"2024-09-10T20:58:52.256190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:45.149805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904189+0000","last_change":"2024-09-10T20:58:47.230068+0000","last_active":"2024-09-10T20:58:53.904189+0000","last_peered":"2024-09-10T20:58:53.904189+0000","last_clean":"2024-09-10T20:58:53.904189+0000","last_became_active":"2024-09-10T20:58:47.229755+0000","last_became_peered":"2024-09-10T20:58:47.229755+0000","last_unstale":"2024-09-10T20:58:53.904189+0000","last_undegraded":"2024-09-10T20:58:53.904189+0000","last_fullsized":"2024-09-10T20:58:53.904189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:04.498018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904966+0000","last_change":"2024-09-10T20:58:46.227149+0000","last_active":"2024-09-10T20:58:53.904966+0000","last_peered":"2024-09-10T20:58:53.904966+0000","last_clean":"2024-09-10T20:58:53.904966+0000","last_became_active":"2024-09-10T20:58:46.226863+0000","last_became_peered":"2024-09-10T20:58:46.226863+0000","last_unstale":"2024-09-10T20:58:53.904966+0000","last_undegraded":"2024-09-10T20:58:53.904966+0000","last_fullsized":"2024-09-10T20:58:53.904966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:28.108849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256521+0000","last_change":"2024-09-10T20:58:51.280288+0000","last_active":"2024-09-10T20:58:52.256521+0000","last_peered":"2024-09-10T20:58:52.256521+0000","last_clean":"2024-09-10T20:58:52.256521+0000","last_became_active":"2024-09-10T20:58:51.279918+0000","last_became_peered":"2024-09-10T20:58:51.279918+0000","last_unstale":"2024-09-10T20:58:52.256521+0000","last_undegraded":"2024-09-10T20:58:52.256521+0000","last_fullsized":"2024-09-10T20:58:52.256521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:36.307037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258303+0000","last_change":"2024-09-10T20:58:47.246700+0000","last_active":"2024-09-10T20:58:52.258303+0000","last_peered":"2024-09-10T20:58:52.258303+0000","last_clean":"2024-09-10T20:58:52.258303+0000","last_became_active":"2024-09-10T20:58:47.246456+0000","last_became_peered":"2024-09-10T20:58:47.246456+0000","last_unstale":"2024-09-10T20:58:52.258303+0000","last_undegraded":"2024-09-10T20:58:52.258303+0000","last_fullsized":"2024-09-10T20:58:52.258303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:08.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316714+0000","last_change":"2024-09-10T20:58:46.224904+0000","last_active":"2024-09-10T20:58:53.316714+0000","last_peered":"2024-09-10T20:58:53.316714+0000","last_clean":"2024-09-10T20:58:53.316714+0000","last_became_active":"2024-09-10T20:58:46.224619+0000","last_became_peered":"2024-09-10T20:58:46.224619+0000","last_unstale":"2024-09-10T20:58:53.316714+0000","last_undegraded":"2024-09-10T20:58:53.316714+0000","last_fullsized":"2024-09-10T20:58:53.316714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:34.223018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256285+0000","last_change":"2024-09-10T20:58:51.274096+0000","last_active":"2024-09-10T20:58:52.256285+0000","last_peered":"2024-09-10T20:58:52.256285+0000","last_clean":"2024-09-10T20:58:52.256285+0000","last_became_active":"2024-09-10T20:58:51.273636+0000","last_became_peered":"2024-09-10T20:58:51.273636+0000","last_unstale":"2024-09-10T20:58:52.256285+0000","last_undegraded":"2024-09-10T20:58:52.256285+0000","last_fullsized":"2024-09-10T20:58:52.256285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:05.160808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258649+0000","last_change":"2024-09-10T20:58:47.228381+0000","last_active":"2024-09-10T20:58:52.258649+0000","last_peered":"2024-09-10T20:58:52.258649+0000","last_clean":"2024-09-10T20:58:52.258649+0000","last_became_active":"2024-09-10T20:58:47.228080+0000","last_became_peered":"2024-09-10T20:58:47.228080+0000","last_unstale":"2024-09-10T20:58:52.258649+0000","last_undegraded":"2024-09-10T20:58:52.258649+0000","last_fullsized":"2024-09-10T20:58:52.258649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:39.726666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316453+0000","last_change":"2024-09-10T20:58:46.226903+0000","last_active":"2024-09-10T20:58:53.316453+0000","last_peered":"2024-09-10T20:58:53.316453+0000","last_clean":"2024-09-10T20:58:53.316453+0000","last_became_active":"2024-09-10T20:58:46.226649+0000","last_became_peered":"2024-09-10T20:58:46.226649+0000","last_unstale":"2024-09-10T20:58:53.316453+0000","last_undegraded":"2024-09-10T20:58:53.316453+0000","last_fullsized":"2024-09-10T20:58:53.316453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:14.748746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317126+0000","last_change":"2024-09-10T20:58:51.554471+0000","last_active":"2024-09-10T20:58:53.317126+0000","last_peered":"2024-09-10T20:58:53.317126+0000","last_clean":"2024-09-10T20:58:53.317126+0000","last_became_active":"2024-09-10T20:58:51.554080+0000","last_became_peered":"2024-09-10T20:58:51.554080+0000","last_unstale":"2024-09-10T20:58:53.317126+0000","last_undegraded":"2024-09-10T20:58:53.317126+0000","last_fullsized":"2024-09-10T20:58:53.317126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:24.547698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256775+0000","last_change":"2024-09-10T20:58:47.234236+0000","last_active":"2024-09-10T20:58:52.256775+0000","last_peered":"2024-09-10T20:58:52.256775+0000","last_clean":"2024-09-10T20:58:52.256775+0000","last_became_active":"2024-09-10T20:58:47.233924+0000","last_became_peered":"2024-09-10T20:58:47.233924+0000","last_unstale":"2024-09-10T20:58:52.256775+0000","last_undegraded":"2024-09-10T20:58:52.256775+0000","last_fullsized":"2024-09-10T20:58:52.256775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:38.992405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903782+0000","last_change":"2024-09-10T20:58:46.221112+0000","last_active":"2024-09-10T20:58:53.903782+0000","last_peered":"2024-09-10T20:58:53.903782+0000","last_clean":"2024-09-10T20:58:53.903782+0000","last_became_active":"2024-09-10T20:58:46.220837+0000","last_became_peered":"2024-09-10T20:58:46.220837+0000","last_unstale":"2024-09-10T20:58:53.903782+0000","last_undegraded":"2024-09-10T20:58:53.903782+0000","last_fullsized":"2024-09-10T20:58:53.903782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:27.129152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256427+0000","last_change":"2024-09-10T20:58:51.274965+0000","last_active":"2024-09-10T20:58:52.256427+0000","last_peered":"2024-09-10T20:58:52.256427+0000","last_clean":"2024-09-10T20:58:52.256427+0000","last_became_active":"2024-09-10T20:58:51.274557+0000","last_became_peered":"2024-09-10T20:58:51.274557+0000","last_unstale":"2024-09-10T20:58:52.256427+0000","last_undegraded":"2024-09-10T20:58:52.256427+0000","last_fullsized":"2024-09-10T20:58:52.256427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:16.787797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904100+0000","last_change":"2024-09-10T20:58:47.246893+0000","last_active":"2024-09-10T20:58:53.904100+0000","last_peered":"2024-09-10T20:58:53.904100+0000","last_clean":"2024-09-10T20:58:53.904100+0000","last_became_active":"2024-09-10T20:58:47.245226+0000","last_became_peered":"2024-09-10T20:58:47.245226+0000","last_unstale":"2024-09-10T20:58:53.904100+0000","last_undegraded":"2024-09-10T20:58:53.904100+0000","last_fullsized":"2024-09-10T20:58:53.904100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:58.155371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904645+0000","last_change":"2024-09-10T20:58:46.222772+0000","last_active":"2024-09-10T20:58:53.904645+0000","last_peered":"2024-09-10T20:58:53.904645+0000","last_clean":"2024-09-10T20:58:53.904645+0000","last_became_active":"2024-09-10T20:58:46.222461+0000","last_became_peered":"2024-09-10T20:58:46.222461+0000","last_unstale":"2024-09-10T20:58:53.904645+0000","last_undegraded":"2024-09-10T20:58:53.904645+0000","last_fullsized":"2024-09-10T20:58:53.904645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:28.108849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256282+0000","last_change":"2024-09-10T20:58:51.554529+0000","last_active":"2024-09-10T20:58:52.256282+0000","last_peered":"2024-09-10T20:58:52.256282+0000","last_clean":"2024-09-10T20:58:52.256282+0000","last_became_active":"2024-09-10T20:58:51.554104+0000","last_became_peered":"2024-09-10T20:58:51.554104+0000","last_unstale":"2024-09-10T20:58:52.256282+0000","last_undegraded":"2024-09-10T20:58:52.256282+0000","last_fullsized":"2024-09-10T20:58:52.256282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:18.603248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.259179+0000","last_change":"2024-09-10T20:58:47.249780+0000","last_active":"2024-09-10T20:58:52.259179+0000","last_peered":"2024-09-10T20:58:52.259179+0000","last_clean":"2024-09-10T20:58:52.259179+0000","last_became_active":"2024-09-10T20:58:47.249519+0000","last_became_peered":"2024-09-10T20:58:47.249519+0000","last_unstale":"2024-09-10T20:58:52.259179+0000","last_undegraded":"2024-09-10T20:58:52.259179+0000","last_fullsized":"2024-09-10T20:58:52.259179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:58.169931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316888+0000","last_change":"2024-09-10T20:58:46.225535+0000","last_active":"2024-09-10T20:58:53.316888+0000","last_peered":"2024-09-10T20:58:53.316888+0000","last_clean":"2024-09-10T20:58:53.316888+0000","last_became_active":"2024-09-10T20:58:46.225248+0000","last_became_peered":"2024-09-10T20:58:46.225248+0000","last_unstale":"2024-09-10T20:58:53.316888+0000","last_undegraded":"2024-09-10T20:58:53.316888+0000","last_fullsized":"2024-09-10T20:58:53.316888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:22.880233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258298+0000","last_change":"2024-09-10T20:58:51.559242+0000","last_active":"2024-09-10T20:58:52.258298+0000","last_peered":"2024-09-10T20:58:52.258298+0000","last_clean":"2024-09-10T20:58:52.258298+0000","last_became_active":"2024-09-10T20:58:51.558875+0000","last_became_peered":"2024-09-10T20:58:51.558875+0000","last_unstale":"2024-09-10T20:58:52.258298+0000","last_undegraded":"2024-09-10T20:58:52.258298+0000","last_fullsized":"2024-09-10T20:58:52.258298+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:18.246776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317045+0000","last_change":"2024-09-10T20:58:47.228376+0000","last_active":"2024-09-10T20:58:53.317045+0000","last_peered":"2024-09-10T20:58:53.317045+0000","last_clean":"2024-09-10T20:58:53.317045+0000","last_became_active":"2024-09-10T20:58:47.228086+0000","last_became_peered":"2024-09-10T20:58:47.228086+0000","last_unstale":"2024-09-10T20:58:53.317045+0000","last_undegraded":"2024-09-10T20:58:53.317045+0000","last_fullsized":"2024-09-10T20:58:53.317045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:07.109711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905238+0000","last_change":"2024-09-10T20:58:46.221847+0000","last_active":"2024-09-10T20:58:53.905238+0000","last_peered":"2024-09-10T20:58:53.905238+0000","last_clean":"2024-09-10T20:58:53.905238+0000","last_became_active":"2024-09-10T20:58:46.221571+0000","last_became_peered":"2024-09-10T20:58:46.221571+0000","last_unstale":"2024-09-10T20:58:53.905238+0000","last_undegraded":"2024-09-10T20:58:53.905238+0000","last_fullsized":"2024-09-10T20:58:53.905238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:44.242563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903819+0000","last_change":"2024-09-10T20:58:51.554547+0000","last_active":"2024-09-10T20:58:53.903819+0000","last_peered":"2024-09-10T20:58:53.903819+0000","last_clean":"2024-09-10T20:58:53.903819+0000","last_became_active":"2024-09-10T20:58:51.554197+0000","last_became_peered":"2024-09-10T20:58:51.554197+0000","last_unstale":"2024-09-10T20:58:53.903819+0000","last_undegraded":"2024-09-10T20:58:53.903819+0000","last_fullsized":"2024-09-10T20:58:53.903819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:25.499034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256428+0000","last_change":"2024-09-10T20:58:47.243797+0000","last_active":"2024-09-10T20:58:52.256428+0000","last_peered":"2024-09-10T20:58:52.256428+0000","last_clean":"2024-09-10T20:58:52.256428+0000","last_became_active":"2024-09-10T20:58:47.243539+0000","last_became_peered":"2024-09-10T20:58:47.243539+0000","last_unstale":"2024-09-10T20:58:52.256428+0000","last_undegraded":"2024-09-10T20:58:52.256428+0000","last_fullsized":"2024-09-10T20:58:52.256428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:30.226963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317072+0000","last_change":"2024-09-10T20:58:46.221373+0000","last_active":"2024-09-10T20:58:53.317072+0000","last_peered":"2024-09-10T20:58:53.317072+0000","last_clean":"2024-09-10T20:58:53.317072+0000","last_became_active":"2024-09-10T20:58:46.221104+0000","last_became_peered":"2024-09-10T20:58:46.221104+0000","last_unstale":"2024-09-10T20:58:53.317072+0000","last_undegraded":"2024-09-10T20:58:53.317072+0000","last_fullsized":"2024-09-10T20:58:53.317072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:29.110234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316552+0000","last_change":"2024-09-10T20:58:51.278666+0000","last_active":"2024-09-10T20:58:53.316552+0000","last_peered":"2024-09-10T20:58:53.316552+0000","last_clean":"2024-09-10T20:58:53.316552+0000","last_became_active":"2024-09-10T20:58:51.278033+0000","last_became_peered":"2024-09-10T20:58:51.278033+0000","last_unstale":"2024-09-10T20:58:53.316552+0000","last_undegraded":"2024-09-10T20:58:53.316552+0000","last_fullsized":"2024-09-10T20:58:53.316552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:59.252455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257022+0000","last_change":"2024-09-10T20:58:47.246884+0000","last_active":"2024-09-10T20:58:52.257022+0000","last_peered":"2024-09-10T20:58:52.257022+0000","last_clean":"2024-09-10T20:58:52.257022+0000","last_became_active":"2024-09-10T20:58:47.246617+0000","last_became_peered":"2024-09-10T20:58:47.246617+0000","last_unstale":"2024-09-10T20:58:52.257022+0000","last_undegraded":"2024-09-10T20:58:52.257022+0000","last_fullsized":"2024-09-10T20:58:52.257022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:32.318955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256070+0000","last_change":"2024-09-10T20:58:46.225503+0000","last_active":"2024-09-10T20:58:52.256070+0000","last_peered":"2024-09-10T20:58:52.256070+0000","last_clean":"2024-09-10T20:58:52.256070+0000","last_became_active":"2024-09-10T20:58:46.225088+0000","last_became_peered":"2024-09-10T20:58:46.225088+0000","last_unstale":"2024-09-10T20:58:52.256070+0000","last_undegraded":"2024-09-10T20:58:52.256070+0000","last_fullsized":"2024-09-10T20:58:52.256070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:36.394527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317025+0000","last_change":"2024-09-10T20:58:51.277749+0000","last_active":"2024-09-10T20:58:53.317025+0000","last_peered":"2024-09-10T20:58:53.317025+0000","last_clean":"2024-09-10T20:58:53.317025+0000","last_became_active":"2024-09-10T20:58:51.277142+0000","last_became_peered":"2024-09-10T20:58:51.277142+0000","last_unstale":"2024-09-10T20:58:53.317025+0000","last_undegraded":"2024-09-10T20:58:53.317025+0000","last_fullsized":"2024-09-10T20:58:53.317025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:42.740546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316380+0000","last_change":"2024-09-10T20:58:47.228660+0000","last_active":"2024-09-10T20:58:53.316380+0000","last_peered":"2024-09-10T20:58:53.316380+0000","last_clean":"2024-09-10T20:58:53.316380+0000","last_became_active":"2024-09-10T20:58:47.228218+0000","last_became_peered":"2024-09-10T20:58:47.228218+0000","last_unstale":"2024-09-10T20:58:53.316380+0000","last_undegraded":"2024-09-10T20:58:53.316380+0000","last_fullsized":"2024-09-10T20:58:53.316380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:51.280929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903422+0000","last_change":"2024-09-10T20:58:46.221228+0000","last_active":"2024-09-10T20:58:53.903422+0000","last_peered":"2024-09-10T20:58:53.903422+0000","last_clean":"2024-09-10T20:58:53.903422+0000","last_became_active":"2024-09-10T20:58:46.220898+0000","last_became_peered":"2024-09-10T20:58:46.220898+0000","last_unstale":"2024-09-10T20:58:53.903422+0000","last_undegraded":"2024-09-10T20:58:53.903422+0000","last_fullsized":"2024-09-10T20:58:53.903422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:18.355281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317144+0000","last_change":"2024-09-10T20:58:51.274949+0000","last_active":"2024-09-10T20:58:53.317144+0000","last_peered":"2024-09-10T20:58:53.317144+0000","last_clean":"2024-09-10T20:58:53.317144+0000","last_became_active":"2024-09-10T20:58:51.274579+0000","last_became_peered":"2024-09-10T20:58:51.274579+0000","last_unstale":"2024-09-10T20:58:53.317144+0000","last_undegraded":"2024-09-10T20:58:53.317144+0000","last_fullsized":"2024-09-10T20:58:53.317144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:50.176038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903242+0000","last_change":"2024-09-10T20:58:47.240914+0000","last_active":"2024-09-10T20:58:53.903242+0000","last_peered":"2024-09-10T20:58:53.903242+0000","last_clean":"2024-09-10T20:58:53.903242+0000","last_became_active":"2024-09-10T20:58:47.240250+0000","last_became_peered":"2024-09-10T20:58:47.240250+0000","last_unstale":"2024-09-10T20:58:53.903242+0000","last_undegraded":"2024-09-10T20:58:53.903242+0000","last_fullsized":"2024-09-10T20:58:53.903242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:46.184593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903948+0000","last_change":"2024-09-10T20:58:46.224644+0000","last_active":"2024-09-10T20:58:53.903948+0000","last_peered":"2024-09-10T20:58:53.903948+0000","last_clean":"2024-09-10T20:58:53.903948+0000","last_became_active":"2024-09-10T20:58:46.224329+0000","last_became_peered":"2024-09-10T20:58:46.224329+0000","last_unstale":"2024-09-10T20:58:53.903948+0000","last_undegraded":"2024-09-10T20:58:53.903948+0000","last_fullsized":"2024-09-10T20:58:53.903948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:01.590603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904042+0000","last_change":"2024-09-10T20:58:51.554468+0000","last_active":"2024-09-10T20:58:53.904042+0000","last_peered":"2024-09-10T20:58:53.904042+0000","last_clean":"2024-09-10T20:58:53.904042+0000","last_became_active":"2024-09-10T20:58:51.554092+0000","last_became_peered":"2024-09-10T20:58:51.554092+0000","last_unstale":"2024-09-10T20:58:53.904042+0000","last_undegraded":"2024-09-10T20:58:53.904042+0000","last_fullsized":"2024-09-10T20:58:53.904042+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:48.958938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258417+0000","last_change":"2024-09-10T20:58:47.242758+0000","last_active":"2024-09-10T20:58:52.258417+0000","last_peered":"2024-09-10T20:58:52.258417+0000","last_clean":"2024-09-10T20:58:52.258417+0000","last_became_active":"2024-09-10T20:58:47.242439+0000","last_became_peered":"2024-09-10T20:58:47.242439+0000","last_unstale":"2024-09-10T20:58:52.258417+0000","last_undegraded":"2024-09-10T20:58:52.258417+0000","last_fullsized":"2024-09-10T20:58:52.258417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:45.212911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255997+0000","last_change":"2024-09-10T20:58:46.222264+0000","last_active":"2024-09-10T20:58:52.255997+0000","last_peered":"2024-09-10T20:58:52.255997+0000","last_clean":"2024-09-10T20:58:52.255997+0000","last_became_active":"2024-09-10T20:58:46.221414+0000","last_became_peered":"2024-09-10T20:58:46.221414+0000","last_unstale":"2024-09-10T20:58:52.255997+0000","last_undegraded":"2024-09-10T20:58:52.255997+0000","last_fullsized":"2024-09-10T20:58:52.255997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:22.019957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":119,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904053+0000","last_change":"2024-09-10T20:58:36.895993+0000","last_active":"2024-09-10T20:58:53.904053+0000","last_peered":"2024-09-10T20:58:53.904053+0000","last_clean":"2024-09-10T20:58:53.904053+0000","last_became_active":"2024-09-10T20:58:36.894988+0000","last_became_peered":"2024-09-10T20:58:36.894988+0000","last_unstale":"2024-09-10T20:58:53.904053+0000","last_undegraded":"2024-09-10T20:58:53.904053+0000","last_fullsized":"2024-09-10T20:58:53.904053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_clean_scrub_stamp":"2024-09-10T20:58:35.588702+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.082528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904143+0000","last_change":"2024-09-10T20:58:51.556599+0000","last_active":"2024-09-10T20:58:53.904143+0000","last_peered":"2024-09-10T20:58:53.904143+0000","last_clean":"2024-09-10T20:58:53.904143+0000","last_became_active":"2024-09-10T20:58:51.556210+0000","last_became_peered":"2024-09-10T20:58:51.556210+0000","last_unstale":"2024-09-10T20:58:53.904143+0000","last_undegraded":"2024-09-10T20:58:53.904143+0000","last_fullsized":"2024-09-10T20:58:53.904143+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:36.307037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904986+0000","last_change":"2024-09-10T20:58:47.245638+0000","last_active":"2024-09-10T20:58:53.904986+0000","last_peered":"2024-09-10T20:58:53.904986+0000","last_clean":"2024-09-10T20:58:53.904986+0000","last_became_active":"2024-09-10T20:58:47.244988+0000","last_became_peered":"2024-09-10T20:58:47.244988+0000","last_unstale":"2024-09-10T20:58:53.904986+0000","last_undegraded":"2024-09-10T20:58:53.904986+0000","last_fullsized":"2024-09-10T20:58:53.904986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:04.445410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903834+0000","last_change":"2024-09-10T20:58:46.228090+0000","last_active":"2024-09-10T20:58:53.903834+0000","last_peered":"2024-09-10T20:58:53.903834+0000","last_clean":"2024-09-10T20:58:53.903834+0000","last_became_active":"2024-09-10T20:58:46.227789+0000","last_became_peered":"2024-09-10T20:58:46.227789+0000","last_unstale":"2024-09-10T20:58:53.903834+0000","last_undegraded":"2024-09-10T20:58:53.903834+0000","last_fullsized":"2024-09-10T20:58:53.903834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:37.064288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904078+0000","last_change":"2024-09-10T20:58:51.554553+0000","last_active":"2024-09-10T20:58:53.904078+0000","last_peered":"2024-09-10T20:58:53.904078+0000","last_clean":"2024-09-10T20:58:53.904078+0000","last_became_active":"2024-09-10T20:58:51.554131+0000","last_became_peered":"2024-09-10T20:58:51.554131+0000","last_unstale":"2024-09-10T20:58:53.904078+0000","last_undegraded":"2024-09-10T20:58:53.904078+0000","last_fullsized":"2024-09-10T20:58:53.904078+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:18.603248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255727+0000","last_change":"2024-09-10T20:58:47.245802+0000","last_active":"2024-09-10T20:58:52.255727+0000","last_peered":"2024-09-10T20:58:52.255727+0000","last_clean":"2024-09-10T20:58:52.255727+0000","last_became_active":"2024-09-10T20:58:47.245543+0000","last_became_peered":"2024-09-10T20:58:47.245543+0000","last_unstale":"2024-09-10T20:58:52.255727+0000","last_undegraded":"2024-09-10T20:58:52.255727+0000","last_fullsized":"2024-09-10T20:58:52.255727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:17.787548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316844+0000","last_change":"2024-09-10T20:58:46.226841+0000","last_active":"2024-09-10T20:58:53.316844+0000","last_peered":"2024-09-10T20:58:53.316844+0000","last_clean":"2024-09-10T20:58:53.316844+0000","last_became_active":"2024-09-10T20:58:46.226610+0000","last_became_peered":"2024-09-10T20:58:46.226610+0000","last_unstale":"2024-09-10T20:58:53.316844+0000","last_undegraded":"2024-09-10T20:58:53.316844+0000","last_fullsized":"2024-09-10T20:58:53.316844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:41.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904125+0000","last_change":"2024-09-10T20:58:51.274667+0000","last_active":"2024-09-10T20:58:53.904125+0000","last_peered":"2024-09-10T20:58:53.904125+0000","last_clean":"2024-09-10T20:58:53.904125+0000","last_became_active":"2024-09-10T20:58:51.274284+0000","last_became_peered":"2024-09-10T20:58:51.274284+0000","last_unstale":"2024-09-10T20:58:53.904125+0000","last_undegraded":"2024-09-10T20:58:53.904125+0000","last_fullsized":"2024-09-10T20:58:53.904125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:36.498874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905137+0000","last_change":"2024-09-10T20:58:47.245680+0000","last_active":"2024-09-10T20:58:53.905137+0000","last_peered":"2024-09-10T20:58:53.905137+0000","last_clean":"2024-09-10T20:58:53.905137+0000","last_became_active":"2024-09-10T20:58:47.245058+0000","last_became_peered":"2024-09-10T20:58:47.245058+0000","last_unstale":"2024-09-10T20:58:53.905137+0000","last_undegraded":"2024-09-10T20:58:53.905137+0000","last_fullsized":"2024-09-10T20:58:53.905137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:36.576513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903742+0000","last_change":"2024-09-10T20:58:46.222762+0000","last_active":"2024-09-10T20:58:53.903742+0000","last_peered":"2024-09-10T20:58:53.903742+0000","last_clean":"2024-09-10T20:58:53.903742+0000","last_became_active":"2024-09-10T20:58:46.222478+0000","last_became_peered":"2024-09-10T20:58:46.222478+0000","last_unstale":"2024-09-10T20:58:53.903742+0000","last_undegraded":"2024-09-10T20:58:53.903742+0000","last_fullsized":"2024-09-10T20:58:53.903742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:20.646322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256423+0000","last_change":"2024-09-10T20:58:51.554839+0000","last_active":"2024-09-10T20:58:52.256423+0000","last_peered":"2024-09-10T20:58:52.256423+0000","last_clean":"2024-09-10T20:58:52.256423+0000","last_became_active":"2024-09-10T20:58:51.554371+0000","last_became_peered":"2024-09-10T20:58:51.554371+0000","last_unstale":"2024-09-10T20:58:52.256423+0000","last_undegraded":"2024-09-10T20:58:52.256423+0000","last_fullsized":"2024-09-10T20:58:52.256423+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:16.242767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905199+0000","last_change":"2024-09-10T20:58:47.239517+0000","last_active":"2024-09-10T20:58:53.905199+0000","last_peered":"2024-09-10T20:58:53.905199+0000","last_clean":"2024-09-10T20:58:53.905199+0000","last_became_active":"2024-09-10T20:58:47.239275+0000","last_became_peered":"2024-09-10T20:58:47.239275+0000","last_unstale":"2024-09-10T20:58:53.905199+0000","last_undegraded":"2024-09-10T20:58:53.905199+0000","last_fullsized":"2024-09-10T20:58:53.905199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:27.143705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256315+0000","last_change":"2024-09-10T20:58:46.226170+0000","last_active":"2024-09-10T20:58:52.256315+0000","last_peered":"2024-09-10T20:58:52.256315+0000","last_clean":"2024-09-10T20:58:52.256315+0000","last_became_active":"2024-09-10T20:58:46.225888+0000","last_became_peered":"2024-09-10T20:58:46.225888+0000","last_unstale":"2024-09-10T20:58:52.256315+0000","last_undegraded":"2024-09-10T20:58:52.256315+0000","last_fullsized":"2024-09-10T20:58:52.256315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:27.108377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255461+0000","last_change":"2024-09-10T20:58:51.276340+0000","last_active":"2024-09-10T20:58:52.255461+0000","last_peered":"2024-09-10T20:58:52.255461+0000","last_clean":"2024-09-10T20:58:52.255461+0000","last_became_active":"2024-09-10T20:58:51.275903+0000","last_became_peered":"2024-09-10T20:58:51.275903+0000","last_unstale":"2024-09-10T20:58:52.255461+0000","last_undegraded":"2024-09-10T20:58:52.255461+0000","last_fullsized":"2024-09-10T20:58:52.255461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:04.361773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904252+0000","last_change":"2024-09-10T20:58:47.237276+0000","last_active":"2024-09-10T20:58:53.904252+0000","last_peered":"2024-09-10T20:58:53.904252+0000","last_clean":"2024-09-10T20:58:53.904252+0000","last_became_active":"2024-09-10T20:58:47.235334+0000","last_became_peered":"2024-09-10T20:58:47.235334+0000","last_unstale":"2024-09-10T20:58:53.904252+0000","last_undegraded":"2024-09-10T20:58:53.904252+0000","last_fullsized":"2024-09-10T20:58:53.904252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:58.169931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903190+0000","last_change":"2024-09-10T20:58:46.224350+0000","last_active":"2024-09-10T20:58:53.903190+0000","last_peered":"2024-09-10T20:58:53.903190+0000","last_clean":"2024-09-10T20:58:53.903190+0000","last_became_active":"2024-09-10T20:58:46.223931+0000","last_became_peered":"2024-09-10T20:58:46.223931+0000","last_unstale":"2024-09-10T20:58:53.903190+0000","last_undegraded":"2024-09-10T20:58:53.903190+0000","last_fullsized":"2024-09-10T20:58:53.903190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:22.019957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903918+0000","last_change":"2024-09-10T20:58:51.270969+0000","last_active":"2024-09-10T20:58:53.903918+0000","last_peered":"2024-09-10T20:58:53.903918+0000","last_clean":"2024-09-10T20:58:53.903918+0000","last_became_active":"2024-09-10T20:58:51.270229+0000","last_became_peered":"2024-09-10T20:58:51.270229+0000","last_unstale":"2024-09-10T20:58:53.903918+0000","last_undegraded":"2024-09-10T20:58:53.903918+0000","last_fullsized":"2024-09-10T20:58:53.903918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:35.803700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903940+0000","last_change":"2024-09-10T20:58:47.247096+0000","last_active":"2024-09-10T20:58:53.903940+0000","last_peered":"2024-09-10T20:58:53.903940+0000","last_clean":"2024-09-10T20:58:53.903940+0000","last_became_active":"2024-09-10T20:58:47.246797+0000","last_became_peered":"2024-09-10T20:58:47.246797+0000","last_unstale":"2024-09-10T20:58:53.903940+0000","last_undegraded":"2024-09-10T20:58:53.903940+0000","last_fullsized":"2024-09-10T20:58:53.903940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:18.950297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904318+0000","last_change":"2024-09-10T20:58:46.226890+0000","last_active":"2024-09-10T20:58:53.904318+0000","last_peered":"2024-09-10T20:58:53.904318+0000","last_clean":"2024-09-10T20:58:53.904318+0000","last_became_active":"2024-09-10T20:58:46.226615+0000","last_became_peered":"2024-09-10T20:58:46.226615+0000","last_unstale":"2024-09-10T20:58:53.904318+0000","last_undegraded":"2024-09-10T20:58:53.904318+0000","last_fullsized":"2024-09-10T20:58:53.904318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:08.671158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.259085+0000","last_change":"2024-09-10T20:58:51.556517+0000","last_active":"2024-09-10T20:58:52.259085+0000","last_peered":"2024-09-10T20:58:52.259085+0000","last_clean":"2024-09-10T20:58:52.259085+0000","last_became_active":"2024-09-10T20:58:51.556152+0000","last_became_peered":"2024-09-10T20:58:51.556152+0000","last_unstale":"2024-09-10T20:58:52.259085+0000","last_undegraded":"2024-09-10T20:58:52.259085+0000","last_fullsized":"2024-09-10T20:58:52.259085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:46.913894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904720+0000","last_change":"2024-09-10T20:58:47.245297+0000","last_active":"2024-09-10T20:58:53.904720+0000","last_peered":"2024-09-10T20:58:53.904720+0000","last_clean":"2024-09-10T20:58:53.904720+0000","last_became_active":"2024-09-10T20:58:47.244947+0000","last_became_peered":"2024-09-10T20:58:47.244947+0000","last_unstale":"2024-09-10T20:58:53.904720+0000","last_undegraded":"2024-09-10T20:58:53.904720+0000","last_fullsized":"2024-09-10T20:58:53.904720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:25.115061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255795+0000","last_change":"2024-09-10T20:58:46.229169+0000","last_active":"2024-09-10T20:58:52.255795+0000","last_peered":"2024-09-10T20:58:52.255795+0000","last_clean":"2024-09-10T20:58:52.255795+0000","last_became_active":"2024-09-10T20:58:46.228622+0000","last_became_peered":"2024-09-10T20:58:46.228622+0000","last_unstale":"2024-09-10T20:58:52.255795+0000","last_undegraded":"2024-09-10T20:58:52.255795+0000","last_fullsized":"2024-09-10T20:58:52.255795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:35.810342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258536+0000","last_change":"2024-09-10T20:58:51.554436+0000","last_active":"2024-09-10T20:58:52.258536+0000","last_peered":"2024-09-10T20:58:52.258536+0000","last_clean":"2024-09-10T20:58:52.258536+0000","last_became_active":"2024-09-10T20:58:51.554038+0000","last_became_peered":"2024-09-10T20:58:51.554038+0000","last_unstale":"2024-09-10T20:58:52.258536+0000","last_undegraded":"2024-09-10T20:58:52.258536+0000","last_fullsized":"2024-09-10T20:58:52.258536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:33.352140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903833+0000","last_change":"2024-09-10T20:58:47.247094+0000","last_active":"2024-09-10T20:58:53.903833+0000","last_peered":"2024-09-10T20:58:53.903833+0000","last_clean":"2024-09-10T20:58:53.903833+0000","last_became_active":"2024-09-10T20:58:47.246095+0000","last_became_peered":"2024-09-10T20:58:47.246095+0000","last_unstale":"2024-09-10T20:58:53.903833+0000","last_undegraded":"2024-09-10T20:58:53.903833+0000","last_fullsized":"2024-09-10T20:58:53.903833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:39.726666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904184+0000","last_change":"2024-09-10T20:58:46.226623+0000","last_active":"2024-09-10T20:58:53.904184+0000","last_peered":"2024-09-10T20:58:53.904184+0000","last_clean":"2024-09-10T20:58:53.904184+0000","last_became_active":"2024-09-10T20:58:46.226326+0000","last_became_peered":"2024-09-10T20:58:46.226326+0000","last_unstale":"2024-09-10T20:58:53.904184+0000","last_undegraded":"2024-09-10T20:58:53.904184+0000","last_fullsized":"2024-09-10T20:58:53.904184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:02:21.934804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256251+0000","last_change":"2024-09-10T20:58:51.556678+0000","last_active":"2024-09-10T20:58:52.256251+0000","last_peered":"2024-09-10T20:58:52.256251+0000","last_clean":"2024-09-10T20:58:52.256251+0000","last_became_active":"2024-09-10T20:58:51.556271+0000","last_became_peered":"2024-09-10T20:58:51.556271+0000","last_unstale":"2024-09-10T20:58:52.256251+0000","last_undegraded":"2024-09-10T20:58:52.256251+0000","last_fullsized":"2024-09-10T20:58:52.256251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:02.438361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904916+0000","last_change":"2024-09-10T20:58:47.243793+0000","last_active":"2024-09-10T20:58:53.904916+0000","last_peered":"2024-09-10T20:58:53.904916+0000","last_clean":"2024-09-10T20:58:53.904916+0000","last_became_active":"2024-09-10T20:58:47.243537+0000","last_became_peered":"2024-09-10T20:58:47.243537+0000","last_unstale":"2024-09-10T20:58:53.904916+0000","last_undegraded":"2024-09-10T20:58:53.904916+0000","last_fullsized":"2024-09-10T20:58:53.904916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:04.464193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:56.612638+0000","last_change":"2024-09-10T20:58:46.225562+0000","last_active":"2024-09-10T20:58:56.612638+0000","last_peered":"2024-09-10T20:58:56.612638+0000","last_clean":"2024-09-10T20:58:56.612638+0000","last_became_active":"2024-09-10T20:58:46.225260+0000","last_became_peered":"2024-09-10T20:58:46.225260+0000","last_unstale":"2024-09-10T20:58:56.612638+0000","last_undegraded":"2024-09-10T20:58:56.612638+0000","last_fullsized":"2024-09-10T20:58:56.612638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:31.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316695+0000","last_change":"2024-09-10T20:58:51.278993+0000","last_active":"2024-09-10T20:58:53.316695+0000","last_peered":"2024-09-10T20:58:53.316695+0000","last_clean":"2024-09-10T20:58:53.316695+0000","last_became_active":"2024-09-10T20:58:51.278069+0000","last_became_peered":"2024-09-10T20:58:51.278069+0000","last_unstale":"2024-09-10T20:58:53.316695+0000","last_undegraded":"2024-09-10T20:58:53.316695+0000","last_fullsized":"2024-09-10T20:58:53.316695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:21.828620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257045+0000","last_change":"2024-09-10T20:58:47.235717+0000","last_active":"2024-09-10T20:58:52.257045+0000","last_peered":"2024-09-10T20:58:52.257045+0000","last_clean":"2024-09-10T20:58:52.257045+0000","last_became_active":"2024-09-10T20:58:47.235451+0000","last_became_peered":"2024-09-10T20:58:47.235451+0000","last_unstale":"2024-09-10T20:58:52.257045+0000","last_undegraded":"2024-09-10T20:58:52.257045+0000","last_fullsized":"2024-09-10T20:58:52.257045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:28.134150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316771+0000","last_change":"2024-09-10T20:58:46.226035+0000","last_active":"2024-09-10T20:58:53.316771+0000","last_peered":"2024-09-10T20:58:53.316771+0000","last_clean":"2024-09-10T20:58:53.316771+0000","last_became_active":"2024-09-10T20:58:46.225778+0000","last_became_peered":"2024-09-10T20:58:46.225778+0000","last_unstale":"2024-09-10T20:58:53.316771+0000","last_undegraded":"2024-09-10T20:58:53.316771+0000","last_fullsized":"2024-09-10T20:58:53.316771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:57.175951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256534+0000","last_change":"2024-09-10T20:58:51.274974+0000","last_active":"2024-09-10T20:58:52.256534+0000","last_peered":"2024-09-10T20:58:52.256534+0000","last_clean":"2024-09-10T20:58:52.256534+0000","last_became_active":"2024-09-10T20:58:51.274597+0000","last_became_peered":"2024-09-10T20:58:51.274597+0000","last_unstale":"2024-09-10T20:58:52.256534+0000","last_undegraded":"2024-09-10T20:58:52.256534+0000","last_fullsized":"2024-09-10T20:58:52.256534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:31.184777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904058+0000","last_change":"2024-09-10T20:58:47.245724+0000","last_active":"2024-09-10T20:58:53.904058+0000","last_peered":"2024-09-10T20:58:53.904058+0000","last_clean":"2024-09-10T20:58:53.904058+0000","last_became_active":"2024-09-10T20:58:47.245252+0000","last_became_peered":"2024-09-10T20:58:47.245252+0000","last_unstale":"2024-09-10T20:58:53.904058+0000","last_undegraded":"2024-09-10T20:58:53.904058+0000","last_fullsized":"2024-09-10T20:58:53.904058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:29.836234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255635+0000","last_change":"2024-09-10T20:58:46.229938+0000","last_active":"2024-09-10T20:58:52.255635+0000","last_peered":"2024-09-10T20:58:52.255635+0000","last_clean":"2024-09-10T20:58:52.255635+0000","last_became_active":"2024-09-10T20:58:46.229121+0000","last_became_peered":"2024-09-10T20:58:46.229121+0000","last_unstale":"2024-09-10T20:58:52.255635+0000","last_undegraded":"2024-09-10T20:58:52.255635+0000","last_fullsized":"2024-09-10T20:58:52.255635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:18.355281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258591+0000","last_change":"2024-09-10T20:58:51.280436+0000","last_active":"2024-09-10T20:58:52.258591+0000","last_peered":"2024-09-10T20:58:52.258591+0000","last_clean":"2024-09-10T20:58:52.258591+0000","last_became_active":"2024-09-10T20:58:51.279981+0000","last_became_peered":"2024-09-10T20:58:51.279981+0000","last_unstale":"2024-09-10T20:58:52.258591+0000","last_undegraded":"2024-09-10T20:58:52.258591+0000","last_fullsized":"2024-09-10T20:58:52.258591+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:01.843001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255541+0000","last_change":"2024-09-10T20:58:47.235938+0000","last_active":"2024-09-10T20:58:52.255541+0000","last_peered":"2024-09-10T20:58:52.255541+0000","last_clean":"2024-09-10T20:58:52.255541+0000","last_became_active":"2024-09-10T20:58:47.235628+0000","last_became_peered":"2024-09-10T20:58:47.235628+0000","last_unstale":"2024-09-10T20:58:52.255541+0000","last_undegraded":"2024-09-10T20:58:52.255541+0000","last_fullsized":"2024-09-10T20:58:52.255541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:13.462491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904541+0000","last_change":"2024-09-10T20:58:46.230618+0000","last_active":"2024-09-10T20:58:53.904541+0000","last_peered":"2024-09-10T20:58:53.904541+0000","last_clean":"2024-09-10T20:58:53.904541+0000","last_became_active":"2024-09-10T20:58:46.230289+0000","last_became_peered":"2024-09-10T20:58:46.230289+0000","last_unstale":"2024-09-10T20:58:53.904541+0000","last_undegraded":"2024-09-10T20:58:53.904541+0000","last_fullsized":"2024-09-10T20:58:53.904541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:51.217167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258193+0000","last_change":"2024-09-10T20:58:51.557495+0000","last_active":"2024-09-10T20:58:52.258193+0000","last_peered":"2024-09-10T20:58:52.258193+0000","last_clean":"2024-09-10T20:58:52.258193+0000","last_became_active":"2024-09-10T20:58:51.556492+0000","last_became_peered":"2024-09-10T20:58:51.556492+0000","last_unstale":"2024-09-10T20:58:52.258193+0000","last_undegraded":"2024-09-10T20:58:52.258193+0000","last_fullsized":"2024-09-10T20:58:52.258193+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:48.716121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903643+0000","last_change":"2024-09-10T20:58:51.555588+0000","last_active":"2024-09-10T20:58:53.903643+0000","last_peered":"2024-09-10T20:58:53.903643+0000","last_clean":"2024-09-10T20:58:53.903643+0000","last_became_active":"2024-09-10T20:58:51.555232+0000","last_became_peered":"2024-09-10T20:58:51.555232+0000","last_unstale":"2024-09-10T20:58:53.903643+0000","last_undegraded":"2024-09-10T20:58:53.903643+0000","last_fullsized":"2024-09-10T20:58:53.903643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.486482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316315+0000","last_change":"2024-09-10T20:58:47.239189+0000","last_active":"2024-09-10T20:58:53.316315+0000","last_peered":"2024-09-10T20:58:53.316315+0000","last_clean":"2024-09-10T20:58:53.316315+0000","last_became_active":"2024-09-10T20:58:47.238926+0000","last_became_peered":"2024-09-10T20:58:47.238926+0000","last_unstale":"2024-09-10T20:58:53.316315+0000","last_undegraded":"2024-09-10T20:58:53.316315+0000","last_fullsized":"2024-09-10T20:58:53.316315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:02.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255350+0000","last_change":"2024-09-10T20:58:46.229369+0000","last_active":"2024-09-10T20:58:52.255350+0000","last_peered":"2024-09-10T20:58:52.255350+0000","last_clean":"2024-09-10T20:58:52.255350+0000","last_became_active":"2024-09-10T20:58:46.229094+0000","last_became_peered":"2024-09-10T20:58:46.229094+0000","last_unstale":"2024-09-10T20:58:52.255350+0000","last_undegraded":"2024-09-10T20:58:52.255350+0000","last_fullsized":"2024-09-10T20:58:52.255350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:20.646322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904390+0000","last_change":"2024-09-10T20:58:51.280161+0000","last_active":"2024-09-10T20:58:53.904390+0000","last_peered":"2024-09-10T20:58:53.904390+0000","last_clean":"2024-09-10T20:58:53.904390+0000","last_became_active":"2024-09-10T20:58:51.279714+0000","last_became_peered":"2024-09-10T20:58:51.279714+0000","last_unstale":"2024-09-10T20:58:53.904390+0000","last_undegraded":"2024-09-10T20:58:53.904390+0000","last_fullsized":"2024-09-10T20:58:53.904390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.486482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258577+0000","last_change":"2024-09-10T20:58:47.248033+0000","last_active":"2024-09-10T20:58:52.258577+0000","last_peered":"2024-09-10T20:58:52.258577+0000","last_clean":"2024-09-10T20:58:52.258577+0000","last_became_active":"2024-09-10T20:58:47.247761+0000","last_became_peered":"2024-09-10T20:58:47.247761+0000","last_unstale":"2024-09-10T20:58:52.258577+0000","last_undegraded":"2024-09-10T20:58:52.258577+0000","last_fullsized":"2024-09-10T20:58:52.258577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:04.498018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258357+0000","last_change":"2024-09-10T20:58:46.223392+0000","last_active":"2024-09-10T20:58:52.258357+0000","last_peered":"2024-09-10T20:58:52.258357+0000","last_clean":"2024-09-10T20:58:52.258357+0000","last_became_active":"2024-09-10T20:58:46.223123+0000","last_became_peered":"2024-09-10T20:58:46.223123+0000","last_unstale":"2024-09-10T20:58:52.258357+0000","last_undegraded":"2024-09-10T20:58:52.258357+0000","last_fullsized":"2024-09-10T20:58:52.258357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:16.570128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904433+0000","last_change":"2024-09-10T20:58:46.227651+0000","last_active":"2024-09-10T20:58:53.904433+0000","last_peered":"2024-09-10T20:58:53.904433+0000","last_clean":"2024-09-10T20:58:53.904433+0000","last_became_active":"2024-09-10T20:58:46.226418+0000","last_became_peered":"2024-09-10T20:58:46.226418+0000","last_unstale":"2024-09-10T20:58:53.904433+0000","last_undegraded":"2024-09-10T20:58:53.904433+0000","last_fullsized":"2024-09-10T20:58:53.904433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:02.951532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904158+0000","last_change":"2024-09-10T20:58:47.245394+0000","last_active":"2024-09-10T20:58:53.904158+0000","last_peered":"2024-09-10T20:58:53.904158+0000","last_clean":"2024-09-10T20:58:53.904158+0000","last_became_active":"2024-09-10T20:58:47.243973+0000","last_became_peered":"2024-09-10T20:58:47.243973+0000","last_unstale":"2024-09-10T20:58:53.904158+0000","last_undegraded":"2024-09-10T20:58:53.904158+0000","last_fullsized":"2024-09-10T20:58:53.904158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:54.993134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903643+0000","last_change":"2024-09-10T20:58:51.274539+0000","last_active":"2024-09-10T20:58:53.903643+0000","last_peered":"2024-09-10T20:58:53.903643+0000","last_clean":"2024-09-10T20:58:53.903643+0000","last_became_active":"2024-09-10T20:58:51.274124+0000","last_became_peered":"2024-09-10T20:58:51.274124+0000","last_unstale":"2024-09-10T20:58:53.903643+0000","last_undegraded":"2024-09-10T20:58:53.903643+0000","last_fullsized":"2024-09-10T20:58:53.903643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:20.469415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258431+0000","last_change":"2024-09-10T20:58:46.223353+0000","last_active":"2024-09-10T20:58:52.258431+0000","last_peered":"2024-09-10T20:58:52.258431+0000","last_clean":"2024-09-10T20:58:52.258431+0000","last_became_active":"2024-09-10T20:58:46.223059+0000","last_became_peered":"2024-09-10T20:58:46.223059+0000","last_unstale":"2024-09-10T20:58:52.258431+0000","last_undegraded":"2024-09-10T20:58:52.258431+0000","last_fullsized":"2024-09-10T20:58:52.258431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:02:21.934804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257112+0000","last_change":"2024-09-10T20:58:47.233445+0000","last_active":"2024-09-10T20:58:52.257112+0000","last_peered":"2024-09-10T20:58:52.257112+0000","last_clean":"2024-09-10T20:58:52.257112+0000","last_became_active":"2024-09-10T20:58:47.233157+0000","last_became_peered":"2024-09-10T20:58:47.233157+0000","last_unstale":"2024-09-10T20:58:52.257112+0000","last_undegraded":"2024-09-10T20:58:52.257112+0000","last_fullsized":"2024-09-10T20:58:52.257112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:52.222165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258673+0000","last_change":"2024-09-10T20:58:51.281498+0000","last_active":"2024-09-10T20:58:52.258673+0000","last_peered":"2024-09-10T20:58:52.258673+0000","last_clean":"2024-09-10T20:58:52.258673+0000","last_became_active":"2024-09-10T20:58:51.281027+0000","last_became_peered":"2024-09-10T20:58:51.281027+0000","last_unstale":"2024-09-10T20:58:52.258673+0000","last_undegraded":"2024-09-10T20:58:52.258673+0000","last_fullsized":"2024-09-10T20:58:52.258673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:35.279483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903782+0000","last_change":"2024-09-10T20:58:47.247788+0000","last_active":"2024-09-10T20:58:53.903782+0000","last_peered":"2024-09-10T20:58:53.903782+0000","last_clean":"2024-09-10T20:58:53.903782+0000","last_became_active":"2024-09-10T20:58:47.247513+0000","last_became_peered":"2024-09-10T20:58:47.247513+0000","last_unstale":"2024-09-10T20:58:53.903782+0000","last_undegraded":"2024-09-10T20:58:53.903782+0000","last_fullsized":"2024-09-10T20:58:53.903782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:59.707834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903324+0000","last_change":"2024-09-10T20:58:47.232038+0000","last_active":"2024-09-10T20:58:53.903324+0000","last_peered":"2024-09-10T20:58:53.903324+0000","last_clean":"2024-09-10T20:58:53.903324+0000","last_became_active":"2024-09-10T20:58:47.231590+0000","last_became_peered":"2024-09-10T20:58:47.231590+0000","last_unstale":"2024-09-10T20:58:53.903324+0000","last_undegraded":"2024-09-10T20:58:53.903324+0000","last_fullsized":"2024-09-10T20:58:53.903324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:26.466044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255659+0000","last_change":"2024-09-10T20:58:47.237625+0000","last_active":"2024-09-10T20:58:52.255659+0000","last_peered":"2024-09-10T20:58:52.255659+0000","last_clean":"2024-09-10T20:58:52.255659+0000","last_became_active":"2024-09-10T20:58:47.237247+0000","last_became_peered":"2024-09-10T20:58:47.237247+0000","last_unstale":"2024-09-10T20:58:52.255659+0000","last_undegraded":"2024-09-10T20:58:52.255659+0000","last_fullsized":"2024-09-10T20:58:52.255659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:21.642999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904061+0000","last_change":"2024-09-10T20:58:47.247495+0000","last_active":"2024-09-10T20:58:53.904061+0000","last_peered":"2024-09-10T20:58:53.904061+0000","last_clean":"2024-09-10T20:58:53.904061+0000","last_became_active":"2024-09-10T20:58:47.247049+0000","last_became_peered":"2024-09-10T20:58:47.247049+0000","last_unstale":"2024-09-10T20:58:53.904061+0000","last_undegraded":"2024-09-10T20:58:53.904061+0000","last_fullsized":"2024-09-10T20:58:53.904061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:18.950297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258717+0000","last_change":"2024-09-10T20:58:47.243173+0000","last_active":"2024-09-10T20:58:52.258717+0000","last_peered":"2024-09-10T20:58:52.258717+0000","last_clean":"2024-09-10T20:58:52.258717+0000","last_became_active":"2024-09-10T20:58:47.242521+0000","last_became_peered":"2024-09-10T20:58:47.242521+0000","last_unstale":"2024-09-10T20:58:52.258717+0000","last_undegraded":"2024-09-10T20:58:52.258717+0000","last_fullsized":"2024-09-10T20:58:52.258717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:54.993134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316600+0000","last_change":"2024-09-10T20:58:47.238462+0000","last_active":"2024-09-10T20:58:53.316600+0000","last_peered":"2024-09-10T20:58:53.316600+0000","last_clean":"2024-09-10T20:58:53.316600+0000","last_became_active":"2024-09-10T20:58:47.238166+0000","last_became_peered":"2024-09-10T20:58:47.238166+0000","last_unstale":"2024-09-10T20:58:53.316600+0000","last_undegraded":"2024-09-10T20:58:53.316600+0000","last_fullsized":"2024-09-10T20:58:53.316600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:51.719957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904758+0000","last_change":"2024-09-10T20:58:47.245403+0000","last_active":"2024-09-10T20:58:53.904758+0000","last_peered":"2024-09-10T20:58:53.904758+0000","last_clean":"2024-09-10T20:58:53.904758+0000","last_became_active":"2024-09-10T20:58:47.245125+0000","last_became_peered":"2024-09-10T20:58:47.245125+0000","last_unstale":"2024-09-10T20:58:53.904758+0000","last_undegraded":"2024-09-10T20:58:53.904758+0000","last_fullsized":"2024-09-10T20:58:53.904758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:29.836234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.259179+0000","last_change":"2024-09-10T20:58:47.242826+0000","last_active":"2024-09-10T20:58:52.259179+0000","last_peered":"2024-09-10T20:58:52.259179+0000","last_clean":"2024-09-10T20:58:52.259179+0000","last_became_active":"2024-09-10T20:58:47.242462+0000","last_became_peered":"2024-09-10T20:58:47.242462+0000","last_unstale":"2024-09-10T20:58:52.259179+0000","last_undegraded":"2024-09-10T20:58:52.259179+0000","last_fullsized":"2024-09-10T20:58:52.259179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:33.221761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255216+0000","last_change":"2024-09-10T20:58:47.236478+0000","last_active":"2024-09-10T20:58:52.255216+0000","last_peered":"2024-09-10T20:58:52.255216+0000","last_clean":"2024-09-10T20:58:52.255216+0000","last_became_active":"2024-09-10T20:58:47.236141+0000","last_became_peered":"2024-09-10T20:58:47.236141+0000","last_unstale":"2024-09-10T20:58:52.255216+0000","last_undegraded":"2024-09-10T20:58:52.255216+0000","last_fullsized":"2024-09-10T20:58:52.255216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:51.719957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258257+0000","last_change":"2024-09-10T20:58:47.233386+0000","last_active":"2024-09-10T20:58:52.258257+0000","last_peered":"2024-09-10T20:58:52.258257+0000","last_clean":"2024-09-10T20:58:52.258257+0000","last_became_active":"2024-09-10T20:58:47.233110+0000","last_became_peered":"2024-09-10T20:58:47.233110+0000","last_unstale":"2024-09-10T20:58:52.258257+0000","last_undegraded":"2024-09-10T20:58:52.258257+0000","last_fullsized":"2024-09-10T20:58:52.258257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:18.950297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255532+0000","last_change":"2024-09-10T20:58:47.236278+0000","last_active":"2024-09-10T20:58:52.255532+0000","last_peered":"2024-09-10T20:58:52.255532+0000","last_clean":"2024-09-10T20:58:52.255532+0000","last_became_active":"2024-09-10T20:58:47.236005+0000","last_became_peered":"2024-09-10T20:58:47.236005+0000","last_unstale":"2024-09-10T20:58:52.255532+0000","last_undegraded":"2024-09-10T20:58:52.255532+0000","last_fullsized":"2024-09-10T20:58:52.255532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:34:54.893698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903878+0000","last_change":"2024-09-10T20:58:47.247778+0000","last_active":"2024-09-10T20:58:53.903878+0000","last_peered":"2024-09-10T20:58:53.903878+0000","last_clean":"2024-09-10T20:58:53.903878+0000","last_became_active":"2024-09-10T20:58:47.247103+0000","last_became_peered":"2024-09-10T20:58:47.247103+0000","last_unstale":"2024-09-10T20:58:53.903878+0000","last_undegraded":"2024-09-10T20:58:53.903878+0000","last_fullsized":"2024-09-10T20:58:53.903878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:42.546277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904334+0000","last_change":"2024-09-10T20:58:47.230471+0000","last_active":"2024-09-10T20:58:53.904334+0000","last_peered":"2024-09-10T20:58:53.904334+0000","last_clean":"2024-09-10T20:58:53.904334+0000","last_became_active":"2024-09-10T20:58:47.230174+0000","last_became_peered":"2024-09-10T20:58:47.230174+0000","last_unstale":"2024-09-10T20:58:53.904334+0000","last_undegraded":"2024-09-10T20:58:53.904334+0000","last_fullsized":"2024-09-10T20:58:53.904334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:54.301892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905155+0000","last_change":"2024-09-10T20:58:47.245889+0000","last_active":"2024-09-10T20:58:53.905155+0000","last_peered":"2024-09-10T20:58:53.905155+0000","last_clean":"2024-09-10T20:58:53.905155+0000","last_became_active":"2024-09-10T20:58:47.245547+0000","last_became_peered":"2024-09-10T20:58:47.245547+0000","last_unstale":"2024-09-10T20:58:53.905155+0000","last_undegraded":"2024-09-10T20:58:53.905155+0000","last_fullsized":"2024-09-10T20:58:53.905155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:11.073447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903659+0000","last_change":"2024-09-10T20:58:47.234594+0000","last_active":"2024-09-10T20:58:53.903659+0000","last_peered":"2024-09-10T20:58:53.903659+0000","last_clean":"2024-09-10T20:58:53.903659+0000","last_became_active":"2024-09-10T20:58:47.234275+0000","last_became_peered":"2024-09-10T20:58:47.234275+0000","last_unstale":"2024-09-10T20:58:53.903659+0000","last_undegraded":"2024-09-10T20:58:53.903659+0000","last_fullsized":"2024-09-10T20:58:53.903659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:25.490881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256889+0000","last_change":"2024-09-10T20:58:47.236653+0000","last_active":"2024-09-10T20:58:52.256889+0000","last_peered":"2024-09-10T20:58:52.256889+0000","last_clean":"2024-09-10T20:58:52.256889+0000","last_became_active":"2024-09-10T20:58:47.236361+0000","last_became_peered":"2024-09-10T20:58:47.236361+0000","last_unstale":"2024-09-10T20:58:52.256889+0000","last_undegraded":"2024-09-10T20:58:52.256889+0000","last_fullsized":"2024-09-10T20:58:52.256889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:50.078953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255103+0000","last_change":"2024-09-10T20:58:47.247779+0000","last_active":"2024-09-10T20:58:52.255103+0000","last_peered":"2024-09-10T20:58:52.255103+0000","last_clean":"2024-09-10T20:58:52.255103+0000","last_became_active":"2024-09-10T20:58:47.246921+0000","last_became_peered":"2024-09-10T20:58:47.246921+0000","last_unstale":"2024-09-10T20:58:52.255103+0000","last_undegraded":"2024-09-10T20:58:52.255103+0000","last_fullsized":"2024-09-10T20:58:52.255103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:07.109711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316370+0000","last_change":"2024-09-10T20:58:47.233169+0000","last_active":"2024-09-10T20:58:53.316370+0000","last_peered":"2024-09-10T20:58:53.316370+0000","last_clean":"2024-09-10T20:58:53.316370+0000","last_became_active":"2024-09-10T20:58:47.232869+0000","last_became_peered":"2024-09-10T20:58:47.232869+0000","last_unstale":"2024-09-10T20:58:53.316370+0000","last_undegraded":"2024-09-10T20:58:53.316370+0000","last_fullsized":"2024-09-10T20:58:53.316370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:37.216505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903723+0000","last_change":"2024-09-10T20:58:47.241778+0000","last_active":"2024-09-10T20:58:53.903723+0000","last_peered":"2024-09-10T20:58:53.903723+0000","last_clean":"2024-09-10T20:58:53.903723+0000","last_became_active":"2024-09-10T20:58:47.241164+0000","last_became_peered":"2024-09-10T20:58:47.241164+0000","last_unstale":"2024-09-10T20:58:53.903723+0000","last_undegraded":"2024-09-10T20:58:53.903723+0000","last_fullsized":"2024-09-10T20:58:53.903723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:59.707834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258272+0000","last_change":"2024-09-10T20:58:47.234831+0000","last_active":"2024-09-10T20:58:52.258272+0000","last_peered":"2024-09-10T20:58:52.258272+0000","last_clean":"2024-09-10T20:58:52.258272+0000","last_became_active":"2024-09-10T20:58:47.234566+0000","last_became_peered":"2024-09-10T20:58:47.234566+0000","last_unstale":"2024-09-10T20:58:52.258272+0000","last_undegraded":"2024-09-10T20:58:52.258272+0000","last_fullsized":"2024-09-10T20:58:52.258272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:29.836234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257006+0000","last_change":"2024-09-10T20:58:47.231297+0000","last_active":"2024-09-10T20:58:52.257006+0000","last_peered":"2024-09-10T20:58:52.257006+0000","last_clean":"2024-09-10T20:58:52.257006+0000","last_became_active":"2024-09-10T20:58:47.230943+0000","last_became_peered":"2024-09-10T20:58:47.230943+0000","last_unstale":"2024-09-10T20:58:52.257006+0000","last_undegraded":"2024-09-10T20:58:52.257006+0000","last_fullsized":"2024-09-10T20:58:52.257006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:29.113846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904210+0000","last_change":"2024-09-10T20:58:47.232404+0000","last_active":"2024-09-10T20:58:53.904210+0000","last_peered":"2024-09-10T20:58:53.904210+0000","last_clean":"2024-09-10T20:58:53.904210+0000","last_became_active":"2024-09-10T20:58:47.232127+0000","last_became_peered":"2024-09-10T20:58:47.232127+0000","last_unstale":"2024-09-10T20:58:53.904210+0000","last_undegraded":"2024-09-10T20:58:53.904210+0000","last_fullsized":"2024-09-10T20:58:53.904210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:39.726666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904756+0000","last_change":"2024-09-10T20:58:47.231816+0000","last_active":"2024-09-10T20:58:53.904756+0000","last_peered":"2024-09-10T20:58:53.904756+0000","last_clean":"2024-09-10T20:58:53.904756+0000","last_became_active":"2024-09-10T20:58:47.231545+0000","last_became_peered":"2024-09-10T20:58:47.231545+0000","last_unstale":"2024-09-10T20:58:53.904756+0000","last_undegraded":"2024-09-10T20:58:53.904756+0000","last_fullsized":"2024-09-10T20:58:53.904756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:54.993134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905112+0000","last_change":"2024-09-10T20:58:47.246656+0000","last_active":"2024-09-10T20:58:53.905112+0000","last_peered":"2024-09-10T20:58:53.905112+0000","last_clean":"2024-09-10T20:58:53.905112+0000","last_became_active":"2024-09-10T20:58:47.245647+0000","last_became_peered":"2024-09-10T20:58:47.245647+0000","last_unstale":"2024-09-10T20:58:53.905112+0000","last_undegraded":"2024-09-10T20:58:53.905112+0000","last_fullsized":"2024-09-10T20:58:53.905112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:35.158790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258006+0000","last_change":"2024-09-10T20:58:47.235102+0000","last_active":"2024-09-10T20:58:52.258006+0000","last_peered":"2024-09-10T20:58:52.258006+0000","last_clean":"2024-09-10T20:58:52.258006+0000","last_became_active":"2024-09-10T20:58:47.233909+0000","last_became_peered":"2024-09-10T20:58:47.233909+0000","last_unstale":"2024-09-10T20:58:52.258006+0000","last_undegraded":"2024-09-10T20:58:52.258006+0000","last_fullsized":"2024-09-10T20:58:52.258006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:59.707834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904152+0000","last_change":"2024-09-10T20:58:47.242793+0000","last_active":"2024-09-10T20:58:53.904152+0000","last_peered":"2024-09-10T20:58:53.904152+0000","last_clean":"2024-09-10T20:58:53.904152+0000","last_became_active":"2024-09-10T20:58:47.242495+0000","last_became_peered":"2024-09-10T20:58:47.242495+0000","last_unstale":"2024-09-10T20:58:53.904152+0000","last_undegraded":"2024-09-10T20:58:53.904152+0000","last_fullsized":"2024-09-10T20:58:53.904152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:21.457962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255428+0000","last_change":"2024-09-10T20:58:47.244668+0000","last_active":"2024-09-10T20:58:52.255428+0000","last_peered":"2024-09-10T20:58:52.255428+0000","last_clean":"2024-09-10T20:58:52.255428+0000","last_became_active":"2024-09-10T20:58:47.244110+0000","last_became_peered":"2024-09-10T20:58:47.244110+0000","last_unstale":"2024-09-10T20:58:52.255428+0000","last_undegraded":"2024-09-10T20:58:52.255428+0000","last_fullsized":"2024-09-10T20:58:52.255428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:41.633550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904570+0000","last_change":"2024-09-10T20:58:47.241409+0000","last_active":"2024-09-10T20:58:53.904570+0000","last_peered":"2024-09-10T20:58:53.904570+0000","last_clean":"2024-09-10T20:58:53.904570+0000","last_became_active":"2024-09-10T20:58:47.241104+0000","last_became_peered":"2024-09-10T20:58:47.241104+0000","last_unstale":"2024-09-10T20:58:53.904570+0000","last_undegraded":"2024-09-10T20:58:53.904570+0000","last_fullsized":"2024-09-10T20:58:53.904570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:08.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255561+0000","last_change":"2024-09-10T20:58:47.246735+0000","last_active":"2024-09-10T20:58:52.255561+0000","last_peered":"2024-09-10T20:58:52.255561+0000","last_clean":"2024-09-10T20:58:52.255561+0000","last_became_active":"2024-09-10T20:58:47.246491+0000","last_became_peered":"2024-09-10T20:58:47.246491+0000","last_unstale":"2024-09-10T20:58:52.255561+0000","last_undegraded":"2024-09-10T20:58:52.255561+0000","last_fullsized":"2024-09-10T20:58:52.255561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:16.591593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904106+0000","last_change":"2024-09-10T20:58:47.246507+0000","last_active":"2024-09-10T20:58:53.904106+0000","last_peered":"2024-09-10T20:58:53.904106+0000","last_clean":"2024-09-10T20:58:53.904106+0000","last_became_active":"2024-09-10T20:58:47.245963+0000","last_became_peered":"2024-09-10T20:58:47.245963+0000","last_unstale":"2024-09-10T20:58:53.904106+0000","last_undegraded":"2024-09-10T20:58:53.904106+0000","last_fullsized":"2024-09-10T20:58:53.904106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:04.498018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316869+0000","last_change":"2024-09-10T20:58:47.242014+0000","last_active":"2024-09-10T20:58:53.316869+0000","last_peered":"2024-09-10T20:58:53.316869+0000","last_clean":"2024-09-10T20:58:53.316869+0000","last_became_active":"2024-09-10T20:58:47.241727+0000","last_became_peered":"2024-09-10T20:58:47.241727+0000","last_unstale":"2024-09-10T20:58:53.316869+0000","last_undegraded":"2024-09-10T20:58:53.316869+0000","last_fullsized":"2024-09-10T20:58:53.316869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:13.462491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904966+0000","last_change":"2024-09-10T20:58:47.246360+0000","last_active":"2024-09-10T20:58:53.904966+0000","last_peered":"2024-09-10T20:58:53.904966+0000","last_clean":"2024-09-10T20:58:53.904966+0000","last_became_active":"2024-09-10T20:58:47.245363+0000","last_became_peered":"2024-09-10T20:58:47.245363+0000","last_unstale":"2024-09-10T20:58:53.904966+0000","last_undegraded":"2024-09-10T20:58:53.904966+0000","last_fullsized":"2024-09-10T20:58:53.904966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:59.710882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":693653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":102829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":102863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":693709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":262144,"data_stored":102793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:59:07.600 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T20:59:07.600 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-10T20:59:07.601 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T20:59:07.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.732+0000 7fe3cf394640 1 -- 172.21.15.98:0/802473722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c80b4000 msgr2=0x7fe3c80cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.732+0000 7fe3cf394640 1 --2- 172.21.15.98:0/802473722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c80b4000 0x7fe3c80cf040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8075110 tx=0x7fe3c002a530 comp rx=0 tx=0).stop 2024-09-10T20:59:07.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.732+0000 7fe3cf394640 1 -- 172.21.15.98:0/802473722 shutdown_connections 2024-09-10T20:59:07.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.732+0000 7fe3cf394640 1 --2- 172.21.15.98:0/802473722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c80b4000 0x7fe3c80cf040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.732+0000 7fe3cf394640 1 --2- 172.21.15.98:0/802473722 >> v2:172.21.15.190:3300/0 conn(0x7fe3c80b3b70 0x7fe3c80ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.732+0000 7fe3cf394640 1 -- 172.21.15.98:0/802473722 >> 172.21.15.98:0/802473722 conn(0x7fe3c80d0c00 msgr2=0x7fe3c80d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:07.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.732+0000 7fe3cf394640 1 -- 172.21.15.98:0/802473722 shutdown_connections 2024-09-10T20:59:07.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.732+0000 7fe3cf394640 1 -- 172.21.15.98:0/802473722 wait complete. 2024-09-10T20:59:07.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.733+0000 7fe3cf394640 1 Processor -- start 2024-09-10T20:59:07.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.733+0000 7fe3cf394640 1 -- start start 2024-09-10T20:59:07.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cf394640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3c80b3b70 0x7fe3c8147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:07.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cf394640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3c80b4000 0x7fe3c814bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:07.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cf394640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c815e6d0 0x7fe3c8149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:07.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cf394640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c80cf700 con 0x7fe3c815e6d0 2024-09-10T20:59:07.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cf394640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c80cf580 con 0x7fe3c80b4000 2024-09-10T20:59:07.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cf394640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c80cf880 con 0x7fe3c80b3b70 2024-09-10T20:59:07.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cd931640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c815e6d0 0x7fe3c8149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:07.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cd931640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c815e6d0 0x7fe3c8149b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58484/0 (socket says 172.21.15.98:58484) 2024-09-10T20:59:07.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cd931640 1 -- 172.21.15.98:0/3517854625 learned_addr learned my addr 172.21.15.98:0/3517854625 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:07.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cc92f640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3c80b4000 0x7fe3c814bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:07.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cd130640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3c80b3b70 0x7fe3c8147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:07.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cd931640 1 -- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3c80b3b70 msgr2=0x7fe3c8147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cd931640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3c80b3b70 0x7fe3c8147940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cd931640 1 -- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3c80b4000 msgr2=0x7fe3c814bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cd931640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3c80b4000 0x7fe3c814bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cd931640 1 -- 172.21.15.98:0/3517854625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c8147e80 con 0x7fe3c815e6d0 2024-09-10T20:59:07.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cc92f640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3c80b4000 0x7fe3c814bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:07.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.734+0000 7fe3cd931640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c815e6d0 0x7fe3c8149b60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009920 tx=0x7fe3c0032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:07.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.735+0000 7fe3c5ffb640 1 -- 172.21.15.98:0/3517854625 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0040070 con 0x7fe3c815e6d0 2024-09-10T20:59:07.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.735+0000 7fe3c5ffb640 1 -- 172.21.15.98:0/3517854625 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3c0005310 con 0x7fe3c815e6d0 2024-09-10T20:59:07.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.735+0000 7fe3cf394640 1 -- 172.21.15.98:0/3517854625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c8148ac0 con 0x7fe3c815e6d0 2024-09-10T20:59:07.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.735+0000 7fe3c5ffb640 1 -- 172.21.15.98:0/3517854625 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0003800 con 0x7fe3c815e6d0 2024-09-10T20:59:07.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.735+0000 7fe3cf394640 1 -- 172.21.15.98:0/3517854625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c8149080 con 0x7fe3c815e6d0 2024-09-10T20:59:07.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.736+0000 7fe3cf394640 1 -- 172.21.15.98:0/3517854625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c80cfd50 con 0x7fe3c815e6d0 2024-09-10T20:59:07.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.736+0000 7fe3c5ffb640 1 -- 172.21.15.98:0/3517854625 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe3c004c030 con 0x7fe3c815e6d0 2024-09-10T20:59:07.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.737+0000 7fe3c5ffb640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fe3b4077fa0 0x7fe3b407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:07.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.737+0000 7fe3c5ffb640 1 -- 172.21.15.98:0/3517854625 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe3c00bb2d0 con 0x7fe3c815e6d0 2024-09-10T20:59:07.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.737+0000 7fe3cd130640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fe3b4077fa0 0x7fe3b407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:07.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.738+0000 7fe3cd130640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fe3b4077fa0 0x7fe3b407a460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8002020 tx=0x7fe3b8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:07.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.741+0000 7fe3c5ffb640 1 -- 172.21.15.98:0/3517854625 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe3c0083a90 con 0x7fe3c815e6d0 2024-09-10T20:59:07.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.914+0000 7fe3cf394640 1 -- 172.21.15.98:0/3517854625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fe3c80d1b10 con 0x7fe3c815e6d0 2024-09-10T20:59:07.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.914+0000 7fe3c5ffb640 1 -- 172.21.15.98:0/3517854625 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v27) ==== 86+0+160 (secure 0 0 0) 0x7fe3c0083390 con 0x7fe3c815e6d0 2024-09-10T20:59:07.918 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:07.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.916+0000 7fe3cf394640 1 -- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fe3b4077fa0 msgr2=0x7fe3b407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.916+0000 7fe3cf394640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fe3b4077fa0 0x7fe3b407a460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8002020 tx=0x7fe3b8009040 comp rx=0 tx=0).stop 2024-09-10T20:59:07.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.916+0000 7fe3cf394640 1 -- 172.21.15.98:0/3517854625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c815e6d0 msgr2=0x7fe3c8149b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:07.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.916+0000 7fe3cf394640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c815e6d0 0x7fe3c8149b60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009920 tx=0x7fe3c0032cb0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.916+0000 7fe3cf394640 1 -- 172.21.15.98:0/3517854625 shutdown_connections 2024-09-10T20:59:07.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.916+0000 7fe3cf394640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fe3b4077fa0 0x7fe3b407a460 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.916+0000 7fe3cf394640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3c815e6d0 0x7fe3c8149b60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.916+0000 7fe3cf394640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3c80b4000 0x7fe3c814bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.916+0000 7fe3cf394640 1 --2- 172.21.15.98:0/3517854625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3c80b3b70 0x7fe3c8147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:07.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.916+0000 7fe3cf394640 1 -- 172.21.15.98:0/3517854625 >> 172.21.15.98:0/3517854625 conn(0x7fe3c80d0c00 msgr2=0x7fe3c815d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:07.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.916+0000 7fe3cf394640 1 -- 172.21.15.98:0/3517854625 shutdown_connections 2024-09-10T20:59:07.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:07.916+0000 7fe3cf394640 1 -- 172.21.15.98:0/3517854625 wait complete. 2024-09-10T20:59:07.931 INFO:teuthology.orchestra.run.smithi098.stdout:[{"version":27,"timestamp":"2024-09-10T20:58:34.614833+0000","name":"","changes":[{"name":"osd.6/osd_mclock_max_capacity_iops_ssd","new_value":"5603.346780"}]}] 2024-09-10T20:59:07.931 INFO:tasks.ceph_manager:config epoch is 27 2024-09-10T20:59:07.931 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T20:59:07.931 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T20:59:07.932 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T20:59:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.027+0000 7fc9d015d640 1 -- 172.21.15.98:0/1360230998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9c80cf210 msgr2=0x7fc9c8072aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.027+0000 7fc9d015d640 1 --2- 172.21.15.98:0/1360230998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9c80cf210 0x7fc9c8072aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8005fd0 tx=0x7fc9b802a540 comp rx=0 tx=0).stop 2024-09-10T20:59:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.027+0000 7fc9d015d640 1 -- 172.21.15.98:0/1360230998 shutdown_connections 2024-09-10T20:59:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.027+0000 7fc9d015d640 1 --2- 172.21.15.98:0/1360230998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9c80cf210 0x7fc9c8072aa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.027+0000 7fc9d015d640 1 --2- 172.21.15.98:0/1360230998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9c8144780 0x7fc9c80b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.027+0000 7fc9d015d640 1 -- 172.21.15.98:0/1360230998 >> 172.21.15.98:0/1360230998 conn(0x7fc9c80b1760 msgr2=0x7fc9c80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.028+0000 7fc9d015d640 1 -- 172.21.15.98:0/1360230998 shutdown_connections 2024-09-10T20:59:08.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.028+0000 7fc9d015d640 1 -- 172.21.15.98:0/1360230998 wait complete. 2024-09-10T20:59:08.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.029+0000 7fc9d015d640 1 Processor -- start 2024-09-10T20:59:08.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.029+0000 7fc9d015d640 1 -- start start 2024-09-10T20:59:08.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.029+0000 7fc9d015d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9c8072680 0x7fc9c80cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:08.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.029+0000 7fc9d015d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9c80cf210 0x7fc9c813af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:08.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.029+0000 7fc9d015d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9c8144780 0x7fc9c80cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:08.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.029+0000 7fc9d015d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c815c9c0 con 0x7fc9c80cf210 2024-09-10T20:59:08.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.029+0000 7fc9d015d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c815c840 con 0x7fc9c8144780 2024-09-10T20:59:08.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.029+0000 7fc9d015d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c815cb40 con 0x7fc9c8072680 2024-09-10T20:59:08.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.029+0000 7fc9cd6f8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9c80cf210 0x7fc9c813af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:08.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9cd6f8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9c80cf210 0x7fc9c813af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58498/0 (socket says 172.21.15.98:58498) 2024-09-10T20:59:08.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9cd6f8640 1 -- 172.21.15.98:0/411893922 learned_addr learned my addr 172.21.15.98:0/411893922 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:08.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9cdef9640 1 --2- 172.21.15.98:0/411893922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9c8072680 0x7fc9c80cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:08.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9ce6fa640 1 --2- 172.21.15.98:0/411893922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9c8144780 0x7fc9c80cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:08.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9cd6f8640 1 -- 172.21.15.98:0/411893922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9c8072680 msgr2=0x7fc9c80cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9cd6f8640 1 --2- 172.21.15.98:0/411893922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9c8072680 0x7fc9c80cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9cd6f8640 1 -- 172.21.15.98:0/411893922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9c8144780 msgr2=0x7fc9c80cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9cd6f8640 1 --2- 172.21.15.98:0/411893922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9c8144780 0x7fc9c80cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9cd6f8640 1 -- 172.21.15.98:0/411893922 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c80ce2b0 con 0x7fc9c80cf210 2024-09-10T20:59:08.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9cd6f8640 1 --2- 172.21.15.98:0/411893922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9c80cf210 0x7fc9c813af80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00c420 tx=0x7fc9bc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:08.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9c67fc640 1 -- 172.21.15.98:0/411893922 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc017070 con 0x7fc9c80cf210 2024-09-10T20:59:08.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9c67fc640 1 -- 172.21.15.98:0/411893922 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9bc0053c0 con 0x7fc9c80cf210 2024-09-10T20:59:08.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9d015d640 1 -- 172.21.15.98:0/411893922 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c813a4a0 con 0x7fc9c80cf210 2024-09-10T20:59:08.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9c67fc640 1 -- 172.21.15.98:0/411893922 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc003850 con 0x7fc9c80cf210 2024-09-10T20:59:08.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.030+0000 7fc9d015d640 1 -- 172.21.15.98:0/411893922 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c813aa60 con 0x7fc9c80cf210 2024-09-10T20:59:08.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.031+0000 7fc9d015d640 1 -- 172.21.15.98:0/411893922 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c80b3da0 con 0x7fc9c80cf210 2024-09-10T20:59:08.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.032+0000 7fc9c67fc640 1 -- 172.21.15.98:0/411893922 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc9bc0039f0 con 0x7fc9c80cf210 2024-09-10T20:59:08.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.032+0000 7fc9c67fc640 1 --2- 172.21.15.98:0/411893922 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fc9b4077fa0 0x7fc9b407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:08.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.032+0000 7fc9c67fc640 1 -- 172.21.15.98:0/411893922 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc9bc099610 con 0x7fc9c80cf210 2024-09-10T20:59:08.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.032+0000 7fc9cdef9640 1 --2- 172.21.15.98:0/411893922 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fc9b4077fa0 0x7fc9b407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:08.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.033+0000 7fc9cdef9640 1 --2- 172.21.15.98:0/411893922 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fc9b4077fa0 0x7fc9b407a460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0005fd0 tx=0x7fc9c00095e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:08.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.036+0000 7fc9c67fc640 1 -- 172.21.15.98:0/411893922 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc9bc061810 con 0x7fc9c80cf210 2024-09-10T20:59:08.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.256+0000 7fc9d015d640 1 -- 172.21.15.98:0/411893922 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc9c80b4d40 con 0x7fc9c80cf210 2024-09-10T20:59:08.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.258+0000 7fc9c67fc640 1 -- 172.21.15.98:0/411893922 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+193248 (secure 0 0 0) 0x7fc9bc066240 con 0x7fc9c80cf210 2024-09-10T20:59:08.265 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:08.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.261+0000 7fc9d015d640 1 -- 172.21.15.98:0/411893922 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fc9b4077fa0 msgr2=0x7fc9b407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.261+0000 7fc9d015d640 1 --2- 172.21.15.98:0/411893922 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fc9b4077fa0 0x7fc9b407a460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0005fd0 tx=0x7fc9c00095e0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.261+0000 7fc9d015d640 1 -- 172.21.15.98:0/411893922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9c80cf210 msgr2=0x7fc9c813af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.261+0000 7fc9d015d640 1 --2- 172.21.15.98:0/411893922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9c80cf210 0x7fc9c813af80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00c420 tx=0x7fc9bc00c8e0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.261+0000 7fc9d015d640 1 -- 172.21.15.98:0/411893922 shutdown_connections 2024-09-10T20:59:08.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.261+0000 7fc9d015d640 1 --2- 172.21.15.98:0/411893922 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fc9b4077fa0 0x7fc9b407a460 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.261+0000 7fc9d015d640 1 --2- 172.21.15.98:0/411893922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9c8144780 0x7fc9c80cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.261+0000 7fc9d015d640 1 --2- 172.21.15.98:0/411893922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9c80cf210 0x7fc9c813af80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.261+0000 7fc9d015d640 1 --2- 172.21.15.98:0/411893922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9c8072680 0x7fc9c80cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.261+0000 7fc9d015d640 1 -- 172.21.15.98:0/411893922 >> 172.21.15.98:0/411893922 conn(0x7fc9c80b1760 msgr2=0x7fc9c80702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:08.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.261+0000 7fc9d015d640 1 -- 172.21.15.98:0/411893922 shutdown_connections 2024-09-10T20:59:08.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.261+0000 7fc9d015d640 1 -- 172.21.15.98:0/411893922 wait complete. 2024-09-10T20:59:08.276 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":5,"flags":0,"active_gid":4112,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":3586222139},{"type":"v1","addr":"172.21.15.190:6801","nonce":3586222139}]},"active_addr":"172.21.15.190:6801/3586222139","active_change":"2024-09-10T20:58:26.897501+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":4215270785}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":94224604}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":3346848798}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.190:0","nonce":2018387116}]}]} 2024-09-10T20:59:08.281 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T20:59:08.281 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T20:59:08.281 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:59:08.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.378+0000 7fbb11b89640 1 -- 172.21.15.98:0/1084800988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb0c070910 msgr2=0x7fbb0c0d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.378+0000 7fbb11b89640 1 --2- 172.21.15.98:0/1084800988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb0c070910 0x7fbb0c0d16f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbb00005fd0 tx=0x7fbb0002aa90 comp rx=0 tx=0).stop 2024-09-10T20:59:08.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.378+0000 7fbb11b89640 1 -- 172.21.15.98:0/1084800988 shutdown_connections 2024-09-10T20:59:08.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.378+0000 7fbb11b89640 1 --2- 172.21.15.98:0/1084800988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb0c0cec00 0x7fbb0c0cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.378+0000 7fbb11b89640 1 --2- 172.21.15.98:0/1084800988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb0c070910 0x7fbb0c0d16f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.378+0000 7fbb11b89640 1 -- 172.21.15.98:0/1084800988 >> 172.21.15.98:0/1084800988 conn(0x7fbb0c0d0c00 msgr2=0x7fbb0c0d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:08.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.378+0000 7fbb11b89640 1 -- 172.21.15.98:0/1084800988 shutdown_connections 2024-09-10T20:59:08.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.378+0000 7fbb11b89640 1 -- 172.21.15.98:0/1084800988 wait complete. 2024-09-10T20:59:08.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.379+0000 7fbb11b89640 1 Processor -- start 2024-09-10T20:59:08.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.379+0000 7fbb11b89640 1 -- start start 2024-09-10T20:59:08.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb11b89640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb0c06ca50 0x7fbb0c1f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:08.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb11b89640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb0c070910 0x7fbb0c156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:08.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb11b89640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0c0cec00 0x7fbb0c159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:08.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb11b89640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c0cf6f0 con 0x7fbb0c06ca50 2024-09-10T20:59:08.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb11b89640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c0cf570 con 0x7fbb0c0cec00 2024-09-10T20:59:08.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb11b89640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbb0c0cf870 con 0x7fbb0c070910 2024-09-10T20:59:08.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb0b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb0c06ca50 0x7fbb0c1f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:08.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb0b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb0c06ca50 0x7fbb0c1f8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58522/0 (socket says 172.21.15.98:58522) 2024-09-10T20:59:08.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb0b7fe640 1 -- 172.21.15.98:0/1023825424 learned_addr learned my addr 172.21.15.98:0/1023825424 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:08.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb0bfff640 1 --2- 172.21.15.98:0/1023825424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0c0cec00 0x7fbb0c159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:08.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb0b7fe640 1 -- 172.21.15.98:0/1023825424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb0c070910 msgr2=0x7fbb0c156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb0affd640 1 --2- 172.21.15.98:0/1023825424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb0c070910 0x7fbb0c156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:08.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb0b7fe640 1 --2- 172.21.15.98:0/1023825424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb0c070910 0x7fbb0c156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb0b7fe640 1 -- 172.21.15.98:0/1023825424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0c0cec00 msgr2=0x7fbb0c159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb0b7fe640 1 --2- 172.21.15.98:0/1023825424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0c0cec00 0x7fbb0c159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb0b7fe640 1 -- 172.21.15.98:0/1023825424 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c1f83c0 con 0x7fbb0c06ca50 2024-09-10T20:59:08.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbb0b7fe640 1 --2- 172.21.15.98:0/1023825424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb0c06ca50 0x7fbb0c1f8e60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c450 tx=0x7fbafc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:08.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbaf3fff640 1 -- 172.21.15.98:0/1023825424 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0052d0 con 0x7fbb0c06ca50 2024-09-10T20:59:08.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.380+0000 7fbaf3fff640 1 -- 172.21.15.98:0/1023825424 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbafc005470 con 0x7fbb0c06ca50 2024-09-10T20:59:08.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.381+0000 7fbb11b89640 1 -- 172.21.15.98:0/1023825424 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0c159c80 con 0x7fbb0c06ca50 2024-09-10T20:59:08.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.381+0000 7fbaf3fff640 1 -- 172.21.15.98:0/1023825424 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc012870 con 0x7fbb0c06ca50 2024-09-10T20:59:08.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.381+0000 7fbb11b89640 1 -- 172.21.15.98:0/1023825424 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0c159f60 con 0x7fbb0c06ca50 2024-09-10T20:59:08.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.381+0000 7fbb11b89640 1 -- 172.21.15.98:0/1023825424 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb0c0cfd90 con 0x7fbb0c06ca50 2024-09-10T20:59:08.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.382+0000 7fbaf3fff640 1 -- 172.21.15.98:0/1023825424 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbafc0035f0 con 0x7fbb0c06ca50 2024-09-10T20:59:08.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.382+0000 7fbaf3fff640 1 --2- 172.21.15.98:0/1023825424 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbaf8077fa0 0x7fbaf807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:08.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.383+0000 7fbaf3fff640 1 -- 172.21.15.98:0/1023825424 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbafc099c80 con 0x7fbb0c06ca50 2024-09-10T20:59:08.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.383+0000 7fbb0affd640 1 --2- 172.21.15.98:0/1023825424 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbaf8077fa0 0x7fbaf807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:08.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.383+0000 7fbb0affd640 1 --2- 172.21.15.98:0/1023825424 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbaf8077fa0 0x7fbaf807a460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbb00035000 tx=0x7fbb00004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:08.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.387+0000 7fbaf3fff640 1 -- 172.21.15.98:0/1023825424 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbafc0623b0 con 0x7fbb0c06ca50 2024-09-10T20:59:08.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.562+0000 7fbb11b89640 1 -- 172.21.15.98:0/1023825424 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb0c0ce7b0 con 0x7fbb0c06ca50 2024-09-10T20:59:08.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.563+0000 7fbaf3fff640 1 -- 172.21.15.98:0/1023825424 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18610 (secure 0 0 0) 0x7fbafc061cb0 con 0x7fbb0c06ca50 2024-09-10T20:59:08.567 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:08.568 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":25,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:58:51.602367+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:58:44.390290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T20:58:45.564099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T20:58:49.250508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:59:08.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.565+0000 7fbb11b89640 1 -- 172.21.15.98:0/1023825424 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbaf8077fa0 msgr2=0x7fbaf807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.565+0000 7fbb11b89640 1 --2- 172.21.15.98:0/1023825424 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbaf8077fa0 0x7fbaf807a460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbb00035000 tx=0x7fbb00004ab0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.565+0000 7fbb11b89640 1 -- 172.21.15.98:0/1023825424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb0c06ca50 msgr2=0x7fbb0c1f8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.565+0000 7fbb11b89640 1 --2- 172.21.15.98:0/1023825424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb0c06ca50 0x7fbb0c1f8e60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c450 tx=0x7fbafc00c910 comp rx=0 tx=0).stop 2024-09-10T20:59:08.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.565+0000 7fbb11b89640 1 -- 172.21.15.98:0/1023825424 shutdown_connections 2024-09-10T20:59:08.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.565+0000 7fbb11b89640 1 --2- 172.21.15.98:0/1023825424 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbaf8077fa0 0x7fbaf807a460 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.565+0000 7fbb11b89640 1 --2- 172.21.15.98:0/1023825424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb0c0cec00 0x7fbb0c159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.565+0000 7fbb11b89640 1 --2- 172.21.15.98:0/1023825424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb0c070910 0x7fbb0c156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.565+0000 7fbb11b89640 1 --2- 172.21.15.98:0/1023825424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb0c06ca50 0x7fbb0c1f8e60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.565+0000 7fbb11b89640 1 -- 172.21.15.98:0/1023825424 >> 172.21.15.98:0/1023825424 conn(0x7fbb0c0d0c00 msgr2=0x7fbb0c15f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:08.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.565+0000 7fbb11b89640 1 -- 172.21.15.98:0/1023825424 shutdown_connections 2024-09-10T20:59:08.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.565+0000 7fbb11b89640 1 -- 172.21.15.98:0/1023825424 wait complete. 2024-09-10T20:59:08.580 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T20:59:08.581 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:59:08.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.677+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/2055207004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bbc004d90 msgr2=0x7f1bbc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.677+0000 7f1bdbd3d640 1 --2- 172.21.15.98:0/2055207004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bbc004d90 0x7f1bbc005210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00c2e0 tx=0x7f1bbc005830 comp rx=0 tx=0).stop 2024-09-10T20:59:08.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.677+0000 7f1bda2da640 1 -- 172.21.15.98:0/2055207004 reap_dead start 2024-09-10T20:59:08.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.677+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/2055207004 shutdown_connections 2024-09-10T20:59:08.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.677+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/2055207004 >> 172.21.15.98:0/2055207004 conn(0x7f1bd40d0c00 msgr2=0x7f1bd40d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:08.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.677+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/2055207004 shutdown_connections 2024-09-10T20:59:08.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.677+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/2055207004 wait complete. 2024-09-10T20:59:08.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.679+0000 7f1bdbd3d640 1 Processor -- start 2024-09-10T20:59:08.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.679+0000 7f1bdbd3d640 1 -- start start 2024-09-10T20:59:08.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.679+0000 7f1bdbd3d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bbc004d90 0x7f1bd41f8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:08.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.679+0000 7f1bdbd3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bd4147010 0x7f1bd41f85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:08.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.679+0000 7f1bdbd3d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bd41475b0 0x7f1bd415b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:08.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.679+0000 7f1bdbd3d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd415efd0 con 0x7f1bbc004d90 2024-09-10T20:59:08.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.679+0000 7f1bdbd3d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd415ee50 con 0x7f1bd4147010 2024-09-10T20:59:08.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.679+0000 7f1bdbd3d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd415f150 con 0x7f1bd41475b0 2024-09-10T20:59:08.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.679+0000 7f1bd9ad9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bbc004d90 0x7f1bd41f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:08.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.679+0000 7f1bd9ad9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bbc004d90 0x7f1bd41f8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58534/0 (socket says 172.21.15.98:58534) 2024-09-10T20:59:08.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.679+0000 7f1bd9ad9640 1 -- 172.21.15.98:0/1983627004 learned_addr learned my addr 172.21.15.98:0/1983627004 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:08.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.680+0000 7f1bda2da640 1 --2- 172.21.15.98:0/1983627004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bd41475b0 0x7f1bd415b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:08.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.680+0000 7f1bd92d8640 1 --2- 172.21.15.98:0/1983627004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bd4147010 0x7f1bd41f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:08.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.680+0000 7f1bd9ad9640 1 -- 172.21.15.98:0/1983627004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bd41475b0 msgr2=0x7f1bd415b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.680+0000 7f1bd9ad9640 1 --2- 172.21.15.98:0/1983627004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bd41475b0 0x7f1bd415b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.680+0000 7f1bd9ad9640 1 -- 172.21.15.98:0/1983627004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bd4147010 msgr2=0x7f1bd41f85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.680+0000 7f1bd9ad9640 1 --2- 172.21.15.98:0/1983627004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bd4147010 0x7f1bd41f85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.680+0000 7f1bd9ad9640 1 -- 172.21.15.98:0/1983627004 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd4155980 con 0x7f1bbc004d90 2024-09-10T20:59:08.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.680+0000 7f1bd9ad9640 1 --2- 172.21.15.98:0/1983627004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bbc004d90 0x7f1bd41f8080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4010c70 tx=0x7f1bc400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:08.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.680+0000 7f1bd27fc640 1 -- 172.21.15.98:0/1983627004 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4003b70 con 0x7f1bbc004d90 2024-09-10T20:59:08.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.680+0000 7f1bd27fc640 1 -- 172.21.15.98:0/1983627004 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bc4003d10 con 0x7f1bbc004d90 2024-09-10T20:59:08.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.680+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/1983627004 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd41f8c60 con 0x7f1bbc004d90 2024-09-10T20:59:08.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.680+0000 7f1bd27fc640 1 -- 172.21.15.98:0/1983627004 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4004280 con 0x7f1bbc004d90 2024-09-10T20:59:08.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.680+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/1983627004 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd406f490 con 0x7f1bbc004d90 2024-09-10T20:59:08.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.681+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/1983627004 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd40b5450 con 0x7f1bbc004d90 2024-09-10T20:59:08.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.682+0000 7f1bd27fc640 1 -- 172.21.15.98:0/1983627004 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1bc400f040 con 0x7f1bbc004d90 2024-09-10T20:59:08.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.682+0000 7f1bd27fc640 1 --2- 172.21.15.98:0/1983627004 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1bc0077fa0 0x7f1bc007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:08.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.682+0000 7f1bd27fc640 1 -- 172.21.15.98:0/1983627004 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1bc409ad20 con 0x7f1bbc004d90 2024-09-10T20:59:08.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.682+0000 7f1bd92d8640 1 --2- 172.21.15.98:0/1983627004 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1bc0077fa0 0x7f1bc007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:08.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.683+0000 7f1bd92d8640 1 --2- 172.21.15.98:0/1983627004 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1bc0077fa0 0x7f1bc007a460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8006990 tx=0x7f1bc8006900 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:08.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.686+0000 7f1bd27fc640 1 -- 172.21.15.98:0/1983627004 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1bc4062e90 con 0x7f1bbc004d90 2024-09-10T20:59:08.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.863+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/1983627004 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bd40b3db0 con 0x7f1bbc004d90 2024-09-10T20:59:08.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.864+0000 7f1bd27fc640 1 -- 172.21.15.98:0/1983627004 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18610 (secure 0 0 0) 0x7f1bc40678c0 con 0x7f1bbc004d90 2024-09-10T20:59:08.869 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:08.869 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":25,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:58:51.602367+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:58:44.390290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T20:58:45.564099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T20:58:49.250508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:59:08.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.867+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/1983627004 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1bc0077fa0 msgr2=0x7f1bc007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.867+0000 7f1bdbd3d640 1 --2- 172.21.15.98:0/1983627004 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1bc0077fa0 0x7f1bc007a460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8006990 tx=0x7f1bc8006900 comp rx=0 tx=0).stop 2024-09-10T20:59:08.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.867+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/1983627004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bbc004d90 msgr2=0x7f1bd41f8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:08.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.867+0000 7f1bdbd3d640 1 --2- 172.21.15.98:0/1983627004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bbc004d90 0x7f1bd41f8080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4010c70 tx=0x7f1bc400e6c0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.867+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/1983627004 shutdown_connections 2024-09-10T20:59:08.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.867+0000 7f1bdbd3d640 1 --2- 172.21.15.98:0/1983627004 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1bc0077fa0 0x7f1bc007a460 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.867+0000 7f1bdbd3d640 1 --2- 172.21.15.98:0/1983627004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bd41475b0 0x7f1bd415b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.867+0000 7f1bdbd3d640 1 --2- 172.21.15.98:0/1983627004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bd4147010 0x7f1bd41f85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.867+0000 7f1bdbd3d640 1 --2- 172.21.15.98:0/1983627004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bbc004d90 0x7f1bd41f8080 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:08.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.867+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/1983627004 >> 172.21.15.98:0/1983627004 conn(0x7f1bd40d0c00 msgr2=0x7f1bd415e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:08.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.867+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/1983627004 shutdown_connections 2024-09-10T20:59:08.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:08.867+0000 7f1bdbd3d640 1 -- 172.21.15.98:0/1983627004 wait complete. 2024-09-10T20:59:08.882 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T20:59:08.883 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T20:59:08.883 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T20:59:08.883 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-10T20:59:08.884 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-10T20:59:08.884 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-10T20:59:08.885 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-10T20:59:08.885 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-10T20:59:09.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.053+0000 7f599c175640 1 -- 172.21.15.98:0/1540155479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5980004c90 msgr2=0x7f59800050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.053+0000 7f599c175640 1 --2- 172.21.15.98:0/1540155479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5980004c90 0x7f59800050b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5990009b30 tx=0x7f5980005830 comp rx=0 tx=0).stop 2024-09-10T20:59:09.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.054+0000 7f51ff59e640 1 -- 172.21.15.98:0/3883302575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51f8054860 msgr2=0x7f51f8054c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.054+0000 7f51ff59e640 1 --2- 172.21.15.98:0/3883302575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51f8054860 0x7f51f8054c40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f51f0005fd0 tx=0x7f51f002a4a0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.054+0000 7f51ff59e640 1 -- 172.21.15.98:0/3883302575 shutdown_connections 2024-09-10T20:59:09.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.054+0000 7f51ff59e640 1 --2- 172.21.15.98:0/3883302575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51f8034650 0x7f51f8008d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.054+0000 7f51ff59e640 1 --2- 172.21.15.98:0/3883302575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51f8054860 0x7f51f8054c40 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f51f0005fd0 tx=0x7f51f002a4a0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.054+0000 7f51ff59e640 1 -- 172.21.15.98:0/3883302575 >> 172.21.15.98:0/3883302575 conn(0x7f51f801e890 msgr2=0x7f51f8055640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.054+0000 7f599c175640 1 -- 172.21.15.98:0/1540155479 shutdown_connections 2024-09-10T20:59:09.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.054+0000 7f599c175640 1 --2- 172.21.15.98:0/1540155479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5980004c90 0x7f59800050b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.054+0000 7f599c175640 1 -- 172.21.15.98:0/1540155479 >> 172.21.15.98:0/1540155479 conn(0x7f59940d1d10 msgr2=0x7f59940d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.054+0000 7f51ff59e640 1 -- 172.21.15.98:0/3883302575 shutdown_connections 2024-09-10T20:59:09.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.054+0000 7f599c175640 1 -- 172.21.15.98:0/1540155479 shutdown_connections 2024-09-10T20:59:09.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.054+0000 7f51ff59e640 1 -- 172.21.15.98:0/3883302575 wait complete. 2024-09-10T20:59:09.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.054+0000 7f599c175640 1 -- 172.21.15.98:0/1540155479 wait complete. 2024-09-10T20:59:09.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.055+0000 7f2e53952640 1 -- 172.21.15.98:0/311083746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c0d0520 msgr2=0x7f2e4c15bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f2e53952640 1 --2- 172.21.15.98:0/311083746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c0d0520 0x7f2e4c15bc50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0b3630 tx=0x7f2e4802a540 comp rx=0 tx=0).stop 2024-09-10T20:59:09.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7ffa35452640 1 -- 172.21.15.98:0/246807233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa18004800 msgr2=0x7ffa18004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7ffa35452640 1 --2- 172.21.15.98:0/246807233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa18004800 0x7ffa18004c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009b20 tx=0x7ffa18005830 comp rx=0 tx=0).stop 2024-09-10T20:59:09.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f63c1f99640 1 -- 172.21.15.98:0/2375831531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b401c890 msgr2=0x7f63b401cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f599c175640 1 Processor -- start 2024-09-10T20:59:09.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f63c1f99640 1 --2- 172.21.15.98:0/2375831531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b401c890 0x7f63b401cc70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f63bc066670 tx=0x7f63bc0a7ce0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7ffa2dffb640 1 -- 172.21.15.98:0/246807233 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24009720 con 0x7ffa18004800 2024-09-10T20:59:09.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f599c175640 1 -- start start 2024-09-10T20:59:09.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7ffa2f7fe640 1 -- 172.21.15.98:0/246807233 reap_dead start 2024-09-10T20:59:09.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7ffa35452640 1 -- 172.21.15.98:0/246807233 shutdown_connections 2024-09-10T20:59:09.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7ffa35452640 1 -- 172.21.15.98:0/246807233 >> 172.21.15.98:0/246807233 conn(0x7ffa300cfaf0 msgr2=0x7ffa300cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f63c1f99640 1 -- 172.21.15.98:0/2375831531 shutdown_connections 2024-09-10T20:59:09.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f4acac0c640 1 -- 172.21.15.98:0/2737738720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac4161560 msgr2=0x7f4ac40d2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f4acac0c640 1 --2- 172.21.15.98:0/2737738720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac4161560 0x7f4ac40d2360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8005fd0 tx=0x7f4ab802a560 comp rx=0 tx=0).stop 2024-09-10T20:59:09.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7ffa35452640 1 -- 172.21.15.98:0/246807233 shutdown_connections 2024-09-10T20:59:09.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f63c1f99640 1 --2- 172.21.15.98:0/2375831531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b40366f0 0x7f63b4009a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f51ff59e640 1 Processor -- start 2024-09-10T20:59:09.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f63c1f99640 1 --2- 172.21.15.98:0/2375831531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b401c890 0x7f63b401cc70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f63c1f99640 1 -- 172.21.15.98:0/2375831531 >> 172.21.15.98:0/2375831531 conn(0x7f63b4054560 msgr2=0x7f63b4054940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f3584fc8640 1 -- 172.21.15.98:0/1295279875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f357801c890 msgr2=0x7f357801cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f51ff59e640 1 -- start start 2024-09-10T20:59:09.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f3584fc8640 1 --2- 172.21.15.98:0/1295279875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f357801c890 0x7f357801cc70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3574009fd0 tx=0x7f357402b750 comp rx=0 tx=0).stop 2024-09-10T20:59:09.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f63c1f99640 1 -- 172.21.15.98:0/2375831531 shutdown_connections 2024-09-10T20:59:09.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7f63c1f99640 1 -- 172.21.15.98:0/2375831531 wait complete. 2024-09-10T20:59:09.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f3584fc8640 1 -- 172.21.15.98:0/1295279875 shutdown_connections 2024-09-10T20:59:09.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f3584fc8640 1 --2- 172.21.15.98:0/1295279875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3578005a60 0x7f3578035fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f3584fc8640 1 --2- 172.21.15.98:0/1295279875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f357801c890 0x7f357801cc70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f3584fc8640 1 -- 172.21.15.98:0/1295279875 >> 172.21.15.98:0/1295279875 conn(0x7f3578054560 msgr2=0x7f3578054940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f2e53952640 1 -- 172.21.15.98:0/311083746 shutdown_connections 2024-09-10T20:59:09.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f2e53952640 1 --2- 172.21.15.98:0/311083746 >> v2:172.21.15.98:3300/0 conn(0x7f2e4c0cdce0 0x7f2e4c0cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f2e53952640 1 --2- 172.21.15.98:0/311083746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c0d0520 0x7f2e4c15bc50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f2e53952640 1 -- 172.21.15.98:0/311083746 >> 172.21.15.98:0/311083746 conn(0x7f2e4c06e480 msgr2=0x7f2e4c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f4acac0c640 1 -- 172.21.15.98:0/2737738720 shutdown_connections 2024-09-10T20:59:09.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f4acac0c640 1 --2- 172.21.15.98:0/2737738720 >> v2:172.21.15.190:3300/0 conn(0x7f4ac40d01b0 0x7f4ac40d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f4acac0c640 1 --2- 172.21.15.98:0/2737738720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac4161560 0x7f4ac40d2360 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f4acac0c640 1 -- 172.21.15.98:0/2737738720 >> 172.21.15.98:0/2737738720 conn(0x7f4ac409d170 msgr2=0x7f4ac40d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.056+0000 7ffa35452640 1 -- 172.21.15.98:0/246807233 wait complete. 2024-09-10T20:59:09.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f4acac0c640 1 -- 172.21.15.98:0/2737738720 shutdown_connections 2024-09-10T20:59:09.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f4acac0c640 1 -- 172.21.15.98:0/2737738720 wait complete. 2024-09-10T20:59:09.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f3584fc8640 1 -- 172.21.15.98:0/1295279875 shutdown_connections 2024-09-10T20:59:09.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f9feeb52640 1 -- 172.21.15.98:0/3014808980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fe815f0f0 msgr2=0x7f9fe80cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f3584fc8640 1 -- 172.21.15.98:0/1295279875 wait complete. 2024-09-10T20:59:09.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f9feeb52640 1 --2- 172.21.15.98:0/3014808980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fe815f0f0 0x7f9fe80cd5d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80b4c50 tx=0x7f9fdc02a5d0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f599c175640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5980004c90 0x7f5994153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f2e53952640 1 -- 172.21.15.98:0/311083746 shutdown_connections 2024-09-10T20:59:09.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f2e53952640 1 -- 172.21.15.98:0/311083746 wait complete. 2024-09-10T20:59:09.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f599c175640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59940c8510 0x7f59940c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f599c175640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59940c4cf0 0x7f59940c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f599c175640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f59940cf8e0 con 0x7f59940c8510 2024-09-10T20:59:09.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f599c175640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f59940709c0 con 0x7f59940c4cf0 2024-09-10T20:59:09.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.057+0000 7f599c175640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f59940cfa60 con 0x7f5980004c90 2024-09-10T20:59:09.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f599a712640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59940c4cf0 0x7f59940c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f5999f11640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5980004c90 0x7f5994153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f51ff59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51f8034650 0x7f51f80e3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f599a712640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59940c4cf0 0x7f59940c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:35090/0 (socket says 172.21.15.98:35090) 2024-09-10T20:59:09.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f51ff59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f8034ba0 0x7f51f800c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f599a712640 1 -- 172.21.15.98:0/1301720631 learned_addr learned my addr 172.21.15.98:0/1301720631 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f5999710640 1 --2- 172.21.15.98:0/1301720631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59940c8510 0x7f59940c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f51ff59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51f80e4ce0 0x7f51f81812c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f51ff59e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f51f80e69c0 con 0x7f51f8034ba0 2024-09-10T20:59:09.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7ffa35452640 1 Processor -- start 2024-09-10T20:59:09.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f5999710640 1 -- 172.21.15.98:0/1301720631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5980004c90 msgr2=0x7f5994153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f5999710640 1 --2- 172.21.15.98:0/1301720631 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5980004c90 0x7f5994153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f5999710640 1 -- 172.21.15.98:0/1301720631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59940c4cf0 msgr2=0x7f59940c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f5999710640 1 --2- 172.21.15.98:0/1301720631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59940c4cf0 0x7f59940c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f5999710640 1 -- 172.21.15.98:0/1301720631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59940c99b0 con 0x7f59940c8510 2024-09-10T20:59:09.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7ffa35452640 1 -- start start 2024-09-10T20:59:09.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f51fdd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f8034ba0 0x7f51f800c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f51fed9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51f80e4ce0 0x7f51f81812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f51fdd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f8034ba0 0x7f51f800c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58600/0 (socket says 172.21.15.98:58600) 2024-09-10T20:59:09.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f9feeb52640 1 -- 172.21.15.98:0/3014808980 shutdown_connections 2024-09-10T20:59:09.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f63c1f99640 1 Processor -- start 2024-09-10T20:59:09.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f9feeb52640 1 --2- 172.21.15.98:0/3014808980 >> v2:172.21.15.98:3300/0 conn(0x7f9fe80cdb10 0x7f9fe80cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f9feeb52640 1 --2- 172.21.15.98:0/3014808980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fe815f0f0 0x7f9fe80cd5d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f9feeb52640 1 --2- 172.21.15.98:0/3014808980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fe80d0520 0x7f9fe815bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f5999710640 1 --2- 172.21.15.98:0/1301720631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59940c8510 0x7f59940c6da0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f598400c500 tx=0x7f598400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f9feeb52640 1 -- 172.21.15.98:0/3014808980 >> 172.21.15.98:0/3014808980 conn(0x7f9fe806e480 msgr2=0x7f9fe80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.058+0000 7f51fdd9b640 1 -- 172.21.15.98:0/159858533 learned_addr learned my addr 172.21.15.98:0/159858533 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f598a7fc640 1 -- 172.21.15.98:0/1301720631 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59840052d0 con 0x7f59940c8510 2024-09-10T20:59:09.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f51ff59e640 1 -- 172.21.15.98:0/159858533 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f51f80e6840 con 0x7f51f80e4ce0 2024-09-10T20:59:09.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f51ff59e640 1 -- 172.21.15.98:0/159858533 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f51f803a540 con 0x7f51f8034650 2024-09-10T20:59:09.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f4acac0c640 1 Processor -- start 2024-09-10T20:59:09.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f599c175640 1 -- 172.21.15.98:0/1301720631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59940c9c90 con 0x7f59940c8510 2024-09-10T20:59:09.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f4acac0c640 1 -- start start 2024-09-10T20:59:09.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f2e53952640 1 Processor -- start 2024-09-10T20:59:09.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f3584fc8640 1 Processor -- start 2024-09-10T20:59:09.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f51fe59c640 1 --2- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51f8034650 0x7f51f80e3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f2e53952640 1 -- start start 2024-09-10T20:59:09.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7f3584fc8640 1 -- start start 2024-09-10T20:59:09.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7f51fdd9b640 1 -- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51f8034650 msgr2=0x7f51f80e3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7f51fdd9b640 1 --2- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51f8034650 0x7f51f80e3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7ffa35452640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa18004800 0x7ffa300c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7f51fdd9b640 1 -- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51f80e4ce0 msgr2=0x7f51f81812c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7f51fdd9b640 1 --2- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51f80e4ce0 0x7f51f81812c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7f51fdd9b640 1 -- 172.21.15.98:0/159858533 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f80e3c40 con 0x7f51f8034ba0 2024-09-10T20:59:09.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7f9feeb52640 1 -- 172.21.15.98:0/3014808980 shutdown_connections 2024-09-10T20:59:09.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7f51fdd9b640 1 --2- 172.21.15.98:0/159858533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f8034ba0 0x7f51f800c110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f520006a990 tx=0x7f520006adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7f9feeb52640 1 -- 172.21.15.98:0/3014808980 wait complete. 2024-09-10T20:59:09.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7ffa35452640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa301f8a10 0x7ffa300acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7ffa35452640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa300abb40 0x7ffa300abff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7ffa35452640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffa300b2df0 con 0x7ffa300abb40 2024-09-10T20:59:09.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7ffa35452640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffa300b2c70 con 0x7ffa301f8a10 2024-09-10T20:59:09.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7ffa35452640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffa300732a0 con 0x7ffa18004800 2024-09-10T20:59:09.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7ffa2effd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa18004800 0x7ffa300c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7ffa2effd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa18004800 0x7ffa300c6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:35978/0 (socket says 172.21.15.98:35978) 2024-09-10T20:59:09.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.060+0000 7ffa2effd640 1 -- 172.21.15.98:0/2662901720 learned_addr learned my addr 172.21.15.98:0/2662901720 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7ffa2e7fc640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa301f8a10 0x7ffa300acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f4acac0c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac40b55a0 0x7f4ac4151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7ffa2f7fe640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa300abb40 0x7ffa300abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f51eeffd640 1 -- 172.21.15.98:0/159858533 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200075630 con 0x7f51f8034ba0 2024-09-10T20:59:09.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f4acac0c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac40d01b0 0x7f4ac4151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f2e53952640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e4c0cdce0 0x7f2e4c1550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f51eeffd640 1 -- 172.21.15.98:0/159858533 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52000757d0 con 0x7f51f8034ba0 2024-09-10T20:59:09.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f51ff59e640 1 -- 172.21.15.98:0/159858533 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f80e3f20 con 0x7f51f8034ba0 2024-09-10T20:59:09.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f4acac0c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ac4161560 0x7f4ac41f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f2e53952640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0d0520 0x7f2e4c1497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f51ff59e640 1 -- 172.21.15.98:0/159858533 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f80e44e0 con 0x7f51f8034ba0 2024-09-10T20:59:09.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f4ac89a8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac40b55a0 0x7f4ac4151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f4ac89a8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac40b55a0 0x7f4ac4151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:35116/0 (socket says 172.21.15.98:35116) 2024-09-10T20:59:09.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f2e516ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e4c0cdce0 0x7f2e4c1550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f4ac3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac40d01b0 0x7f4ac4151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f4ac3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac40d01b0 0x7f4ac4151780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:35984/0 (socket says 172.21.15.98:35984) 2024-09-10T20:59:09.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f2e516ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e4c0cdce0 0x7f2e4c1550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:35986/0 (socket says 172.21.15.98:35986) 2024-09-10T20:59:09.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f4ac89a8640 1 -- 172.21.15.98:0/2743062534 learned_addr learned my addr 172.21.15.98:0/2743062534 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f2e516ee640 1 -- 172.21.15.98:0/3896199643 learned_addr learned my addr 172.21.15.98:0/3896199643 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f51eeffd640 1 -- 172.21.15.98:0/159858533 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200076020 con 0x7f51f8034ba0 2024-09-10T20:59:09.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f3584fc8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3578005a60 0x7f3578049da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f4acac0c640 1 -- 172.21.15.98:0/2743062534 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac40d48d0 con 0x7f4ac4161560 2024-09-10T20:59:09.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f2e53952640 1 --2- 172.21.15.98:0/3896199643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c15f0f0 0x7f2e4c1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f4acac0c640 1 -- 172.21.15.98:0/2743062534 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac40b4c50 con 0x7f4ac40b55a0 2024-09-10T20:59:09.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f4acac0c640 1 -- 172.21.15.98:0/2743062534 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4ac40cf050 con 0x7f4ac40d01b0 2024-09-10T20:59:09.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7ffa2e7fc640 1 -- 172.21.15.98:0/2662901720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa18004800 msgr2=0x7ffa300c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f2e53952640 1 -- 172.21.15.98:0/3896199643 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c0cf720 con 0x7f2e4c0d0520 2024-09-10T20:59:09.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f3584fc8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f357801c890 0x7f35780e0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f2e53952640 1 -- 172.21.15.98:0/3896199643 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c0cf5a0 con 0x7f2e4c15f0f0 2024-09-10T20:59:09.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f2e53952640 1 -- 172.21.15.98:0/3896199643 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4c0cf8a0 con 0x7f2e4c0cdce0 2024-09-10T20:59:09.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f357f7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3578005a60 0x7f3578049da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f2e51eef640 1 --2- 172.21.15.98:0/3896199643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c15f0f0 0x7f2e4c1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f357f7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3578005a60 0x7f3578049da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:35998/0 (socket says 172.21.15.98:35998) 2024-09-10T20:59:09.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f3584fc8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3578036520 0x7f35780e0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f599c175640 1 -- 172.21.15.98:0/1301720631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59940b1a50 con 0x7f59940c8510 2024-09-10T20:59:09.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f598a7fc640 1 -- 172.21.15.98:0/1301720631 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5984005490 con 0x7f59940c8510 2024-09-10T20:59:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f357effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f357801c890 0x7f35780e0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f357effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f357801c890 0x7f35780e0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:35130/0 (socket says 172.21.15.98:35130) 2024-09-10T20:59:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f357f7fe640 1 -- 172.21.15.98:0/1559210875 learned_addr learned my addr 172.21.15.98:0/1559210875 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f3584fc8640 1 -- 172.21.15.98:0/1559210875 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f357800aa60 con 0x7f3578036520 2024-09-10T20:59:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f3584fc8640 1 -- 172.21.15.98:0/1559210875 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f357800bc10 con 0x7f357801c890 2024-09-10T20:59:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f3584fc8640 1 -- 172.21.15.98:0/1559210875 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f357800abe0 con 0x7f3578005a60 2024-09-10T20:59:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7ffa2e7fc640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa18004800 0x7ffa300c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f4ac3fff640 1 -- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac40b55a0 msgr2=0x7f4ac4151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7ffa2e7fc640 1 -- 172.21.15.98:0/2662901720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa300abb40 msgr2=0x7ffa300abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7ffa2e7fc640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa300abb40 0x7ffa300abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f2e51eef640 1 -- 172.21.15.98:0/3896199643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e4c0cdce0 msgr2=0x7f2e4c1550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f2e51eef640 1 --2- 172.21.15.98:0/3896199643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2e4c0cdce0 0x7f2e4c1550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f2e51eef640 1 -- 172.21.15.98:0/3896199643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0d0520 msgr2=0x7f2e4c1497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f2e51eef640 1 --2- 172.21.15.98:0/3896199643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0d0520 0x7f2e4c1497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f2e51eef640 1 -- 172.21.15.98:0/3896199643 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c1526f0 con 0x7f2e4c15f0f0 2024-09-10T20:59:09.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.059+0000 7f63c1f99640 1 -- start start 2024-09-10T20:59:09.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7ffa2e7fc640 1 -- 172.21.15.98:0/2662901720 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa300c49d0 con 0x7ffa301f8a10 2024-09-10T20:59:09.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7ffa2f7fe640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa300abb40 0x7ffa300abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:09.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f357f7fe640 1 -- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f357801c890 msgr2=0x7f35780e0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f357f7fe640 1 --2- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f357801c890 0x7f35780e0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f357f7fe640 1 -- 172.21.15.98:0/1559210875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3578036520 msgr2=0x7f35780e0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f357f7fe640 1 --2- 172.21.15.98:0/1559210875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3578036520 0x7f35780e0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f357f7fe640 1 -- 172.21.15.98:0/1559210875 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35781752e0 con 0x7f3578005a60 2024-09-10T20:59:09.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f63c1f99640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b4005a60 0x7f63b4049830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f63c1f99640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b401c890 0x7f63b40d5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f63c1f99640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40366f0 0x7f63b40d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f63c1f99640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f63b4009f90 con 0x7f63b401c890 2024-09-10T20:59:09.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f63c1f99640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f63b400bcd0 con 0x7f63b40366f0 2024-09-10T20:59:09.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.061+0000 7f63c1f99640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f63b400a110 con 0x7f63b4005a60 2024-09-10T20:59:09.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f4ac3fff640 1 --2- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac40b55a0 0x7f4ac4151240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f4ac3fff640 1 -- 172.21.15.98:0/2743062534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ac4161560 msgr2=0x7f4ac41f7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f4ac3fff640 1 --2- 172.21.15.98:0/2743062534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ac4161560 0x7f4ac41f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f4ac3fff640 1 -- 172.21.15.98:0/2743062534 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac4154fb0 con 0x7f4ac40d01b0 2024-09-10T20:59:09.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f4ac3fff640 1 --2- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac40d01b0 0x7f4ac4151780 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000c3c0 tx=0x7f4ab000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7ffa2e7fc640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa301f8a10 0x7ffa300acb50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffa24032000 tx=0x7ffa24033670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f2e51eef640 1 --2- 172.21.15.98:0/3896199643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c15f0f0 0x7f2e4c1521b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2e44005fd0 tx=0x7f2e4400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f9feeb52640 1 Processor -- start 2024-09-10T20:59:09.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f9feeb52640 1 -- start start 2024-09-10T20:59:09.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.063+0000 7f4ac17fa640 1 -- 172.21.15.98:0/2743062534 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0017070 con 0x7f4ac40d01b0 2024-09-10T20:59:09.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.063+0000 7f598a7fc640 1 -- 172.21.15.98:0/1301720631 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984012890 con 0x7f59940c8510 2024-09-10T20:59:09.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.063+0000 7f4ac17fa640 1 -- 172.21.15.98:0/2743062534 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ab00053c0 con 0x7f4ac40d01b0 2024-09-10T20:59:09.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.063+0000 7f4ac17fa640 1 -- 172.21.15.98:0/2743062534 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0003850 con 0x7f4ac40d01b0 2024-09-10T20:59:09.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.063+0000 7f4acac0c640 1 -- 172.21.15.98:0/2743062534 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac409e1c0 con 0x7f4ac40d01b0 2024-09-10T20:59:09.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.063+0000 7f598a7fc640 1 -- 172.21.15.98:0/1301720631 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5984012af0 con 0x7f59940c8510 2024-09-10T20:59:09.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.063+0000 7ffa177fe640 1 -- 172.21.15.98:0/2662901720 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24038070 con 0x7ffa301f8a10 2024-09-10T20:59:09.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.063+0000 7f4acac0c640 1 -- 172.21.15.98:0/2743062534 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac409fe80 con 0x7f4ac40d01b0 2024-09-10T20:59:09.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.063+0000 7f357f7fe640 1 --2- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3578005a60 0x7f3578049da0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3574009470 tx=0x7f3574005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.063+0000 7ffa177fe640 1 -- 172.21.15.98:0/2662901720 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa24039040 con 0x7ffa301f8a10 2024-09-10T20:59:09.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.063+0000 7ffa35452640 1 -- 172.21.15.98:0/2662901720 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa301f9240 con 0x7ffa301f8a10 2024-09-10T20:59:09.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f599c175640 1 -- 172.21.15.98:0/1301720631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f597c001030 con 0x7f59940c8510 2024-09-10T20:59:09.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f598a7fc640 1 --2- 172.21.15.98:0/1301720631 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5978077f50 0x7f597807a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.063+0000 7ffa177fe640 1 -- 172.21.15.98:0/2662901720 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24033cb0 con 0x7ffa301f8a10 2024-09-10T20:59:09.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.063+0000 7ffa35452640 1 -- 172.21.15.98:0/2662901720 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa301f9800 con 0x7ffa301f8a10 2024-09-10T20:59:09.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f9feeb52640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fe80cdb10 0x7f9fe81f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f9feeb52640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe80d0520 0x7f9fe814d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f9feeb52640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fe815f0f0 0x7f9fe814aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f9feeb52640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe80b4800 con 0x7f9fe80d0520 2024-09-10T20:59:09.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f9feeb52640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe80b4680 con 0x7f9fe815f0f0 2024-09-10T20:59:09.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f9feeb52640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe80cf5a0 con 0x7f9fe80cdb10 2024-09-10T20:59:09.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f356bfff640 1 -- 172.21.15.98:0/1559210875 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35740055f0 con 0x7f3578005a60 2024-09-10T20:59:09.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f9fec8ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fe80cdb10 0x7f9fe81f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f9fec8ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fe80cdb10 0x7f9fe81f8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:36004/0 (socket says 172.21.15.98:36004) 2024-09-10T20:59:09.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f9fec8ee640 1 -- 172.21.15.98:0/1432281821 learned_addr learned my addr 172.21.15.98:0/1432281821 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f4acac0c640 1 -- 172.21.15.98:0/2743062534 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ac4160cc0 con 0x7f4ac40d01b0 2024-09-10T20:59:09.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f2e41ffb640 1 -- 172.21.15.98:0/3896199643 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e440139d0 con 0x7f2e4c15f0f0 2024-09-10T20:59:09.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f356bfff640 1 -- 172.21.15.98:0/1559210875 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3574004050 con 0x7f3578005a60 2024-09-10T20:59:09.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f5999f11640 1 --2- 172.21.15.98:0/1301720631 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5978077f50 0x7f597807a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.065+0000 7f598a7fc640 1 -- 172.21.15.98:0/1301720631 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f598405d7f0 con 0x7f59940c8510 2024-09-10T20:59:09.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.065+0000 7f63c0f97640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b4005a60 0x7f63b4049830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.065+0000 7f63c1798640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40366f0 0x7f63b40d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.065+0000 7f63c1798640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40366f0 0x7f63b40d5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:35144/0 (socket says 172.21.15.98:35144) 2024-09-10T20:59:09.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.065+0000 7f63c1798640 1 -- 172.21.15.98:0/275509787 learned_addr learned my addr 172.21.15.98:0/275509787 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7ffa35452640 1 -- 172.21.15.98:0/2662901720 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa300b4960 con 0x7ffa301f8a10 2024-09-10T20:59:09.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f2e53952640 1 -- 172.21.15.98:0/3896199643 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c1489f0 con 0x7f2e4c15f0f0 2024-09-10T20:59:09.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.065+0000 7f2e53952640 1 -- 172.21.15.98:0/3896199643 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c148fb0 con 0x7f2e4c15f0f0 2024-09-10T20:59:09.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.065+0000 7f2e41ffb640 1 -- 172.21.15.98:0/3896199643 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e44013c60 con 0x7f2e4c15f0f0 2024-09-10T20:59:09.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.062+0000 7f51ff59e640 1 -- 172.21.15.98:0/159858533 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f51e4001030 con 0x7f51f8034ba0 2024-09-10T20:59:09.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f356bfff640 1 -- 172.21.15.98:0/1559210875 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574004480 con 0x7f3578005a60 2024-09-10T20:59:09.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f3584fc8640 1 -- 172.21.15.98:0/1559210875 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35781755c0 con 0x7f3578005a60 2024-09-10T20:59:09.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f3584fc8640 1 -- 172.21.15.98:0/1559210875 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3578175b80 con 0x7f3578005a60 2024-09-10T20:59:09.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.065+0000 7f3584fc8640 1 -- 172.21.15.98:0/1559210875 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35780078b0 con 0x7f3578005a60 2024-09-10T20:59:09.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7ffa177fe640 1 -- 172.21.15.98:0/2662901720 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ffa24061a50 con 0x7ffa301f8a10 2024-09-10T20:59:09.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f9fec8ee640 1 -- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fe815f0f0 msgr2=0x7f9fe814aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f9fec8ee640 1 --2- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fe815f0f0 0x7f9fe814aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f9fec8ee640 1 -- 172.21.15.98:0/1432281821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe80d0520 msgr2=0x7f9fe814d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f9fec8ee640 1 --2- 172.21.15.98:0/1432281821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fe80d0520 0x7f9fe814d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.064+0000 7f9fec8ee640 1 -- 172.21.15.98:0/1432281821 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe814b930 con 0x7f9fe80cdb10 2024-09-10T20:59:09.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.065+0000 7f9fec8ee640 1 --2- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fe80cdb10 0x7f9fe81f8c20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000ebc0 tx=0x7f9fe000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.065+0000 7f9fe57fa640 1 -- 172.21.15.98:0/1432281821 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0010630 con 0x7f9fe80cdb10 2024-09-10T20:59:09.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.065+0000 7f9fe57fa640 1 -- 172.21.15.98:0/1432281821 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fe00108c0 con 0x7f9fe80cdb10 2024-09-10T20:59:09.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.065+0000 7f9fe57fa640 1 -- 172.21.15.98:0/1432281821 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0010cd0 con 0x7f9fe80cdb10 2024-09-10T20:59:09.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f2e41ffb640 1 -- 172.21.15.98:0/3896199643 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4401b690 con 0x7f2e4c15f0f0 2024-09-10T20:59:09.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f51eeffd640 1 -- 172.21.15.98:0/159858533 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f520009b030 con 0x7f51f8034ba0 2024-09-10T20:59:09.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f598a7fc640 1 --2- 172.21.15.98:0/1301720631 >> [v2:172.21.15.98:6824/3415066506,v1:172.21.15.98:6825/3415066506] conn(0x7f59780816d0 0x7f5978083b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f63bbfff640 1 --2- 172.21.15.98:0/275509787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b401c890 0x7f63b40d5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f63c1798640 1 -- 172.21.15.98:0/275509787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b4005a60 msgr2=0x7f63b4049830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f63c1798640 1 --2- 172.21.15.98:0/275509787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63b4005a60 0x7f63b4049830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f63c1798640 1 -- 172.21.15.98:0/275509787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b401c890 msgr2=0x7f63b40d5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f63c1798640 1 --2- 172.21.15.98:0/275509787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b401c890 0x7f63b40d5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f63c1798640 1 -- 172.21.15.98:0/275509787 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b4049fc0 con 0x7f63b40366f0 2024-09-10T20:59:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f63bbfff640 1 --2- 172.21.15.98:0/275509787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b401c890 0x7f63b40d5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f63c1798640 1 --2- 172.21.15.98:0/275509787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40366f0 0x7f63b40d5ac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f63ac007860 tx=0x7f63ac007d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f2e53952640 1 -- 172.21.15.98:0/3896199643 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2e2c001030 con 0x7f2e4c15f0f0 2024-09-10T20:59:09.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.065+0000 7f4ac17fa640 1 -- 172.21.15.98:0/2743062534 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4ab00039f0 con 0x7f4ac40d01b0 2024-09-10T20:59:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f4ac17fa640 1 --2- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4aa8077f70 0x7f4aa807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f4ac17fa640 1 -- 172.21.15.98:0/2743062534 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4ab00995d0 con 0x7f4ac40d01b0 2024-09-10T20:59:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f4ac17fa640 1 --2- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:6818/3616946293,v1:172.21.15.190:6819/3616946293] conn(0x7f4aa8081670 0x7f4aa8083ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f4ac17fa640 1 -- 172.21.15.98:0/2743062534 --> [v2:172.21.15.190:6818/3616946293,v1:172.21.15.190:6819/3616946293] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ab001fe50 con 0x7f4aa8081670 2024-09-10T20:59:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f4ac17fa640 1 -- 172.21.15.98:0/2743062534 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4ab00a0050 con 0x7f4ac40d01b0 2024-09-10T20:59:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f63b97fa640 1 -- 172.21.15.98:0/275509787 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac017030 con 0x7f63b40366f0 2024-09-10T20:59:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f63b97fa640 1 -- 172.21.15.98:0/275509787 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63ac00ae10 con 0x7f63b40366f0 2024-09-10T20:59:09.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f63b97fa640 1 -- 172.21.15.98:0/275509787 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac00db10 con 0x7f63b40366f0 2024-09-10T20:59:09.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f5999f11640 1 --2- 172.21.15.98:0/1301720631 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5978077f50 0x7f597807a410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5990035000 tx=0x7f59900046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f598a7fc640 1 -- 172.21.15.98:0/1301720631 --> [v2:172.21.15.98:6824/3415066506,v1:172.21.15.98:6825/3415066506] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5984013e70 con 0x7f59780816d0 2024-09-10T20:59:09.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f598a7fc640 1 -- 172.21.15.98:0/1301720631 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f598409e060 con 0x7f59940c8510 2024-09-10T20:59:09.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f599a712640 1 --2- 172.21.15.98:0/1301720631 >> [v2:172.21.15.98:6824/3415066506,v1:172.21.15.98:6825/3415066506] conn(0x7f59780816d0 0x7f5978083b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f51eeffd640 1 --2- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f51e0078040 0x7f51e007a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f63c1f99640 1 -- 172.21.15.98:0/275509787 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b404a2a0 con 0x7f63b40366f0 2024-09-10T20:59:09.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f63c1f99640 1 -- 172.21.15.98:0/275509787 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b404a860 con 0x7f63b40366f0 2024-09-10T20:59:09.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f51eeffd640 1 -- 172.21.15.98:0/159858533 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5200114ab0 con 0x7f51f8034ba0 2024-09-10T20:59:09.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f51eeffd640 1 --2- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:6826/2783440087,v1:172.21.15.190:6827/2783440087] conn(0x7f51e00817b0 0x7f51e0083bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f599a712640 1 --2- 172.21.15.98:0/1301720631 >> [v2:172.21.15.98:6824/3415066506,v1:172.21.15.98:6825/3415066506] conn(0x7f59780816d0 0x7f5978083b10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.066+0000 7f9feeb52640 1 -- 172.21.15.98:0/1432281821 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe81f9470 con 0x7f9fe80cdb10 2024-09-10T20:59:09.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f9feeb52640 1 -- 172.21.15.98:0/1432281821 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe81f9a30 con 0x7f9fe80cdb10 2024-09-10T20:59:09.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f51fe59c640 1 --2- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f51e0078040 0x7f51e007a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f4ac91a9640 1 --2- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:6818/3616946293,v1:172.21.15.190:6819/3616946293] conn(0x7f4aa8081670 0x7f4aa8083ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f51eeffd640 1 -- 172.21.15.98:0/159858533 --> [v2:172.21.15.190:6826/2783440087,v1:172.21.15.190:6827/2783440087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51e00842c0 con 0x7f51e00817b0 2024-09-10T20:59:09.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f4ac89a8640 1 --2- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4aa8077f70 0x7f4aa807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f4ac91a9640 1 --2- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:6818/3616946293,v1:172.21.15.190:6819/3616946293] conn(0x7f4aa8081670 0x7f4aa8083ab0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.067+0000 7f51eeffd640 1 -- 172.21.15.98:0/159858533 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f52001192a0 con 0x7f51f8034ba0 2024-09-10T20:59:09.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7f356bfff640 1 -- 172.21.15.98:0/1559210875 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f357404c030 con 0x7f3578005a60 2024-09-10T20:59:09.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7f51fed9d640 1 --2- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:6826/2783440087,v1:172.21.15.190:6827/2783440087] conn(0x7f51e00817b0 0x7f51e0083bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7f63a2ffd640 1 -- 172.21.15.98:0/275509787 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f63b4049d70 con 0x7f63b40366f0 2024-09-10T20:59:09.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7f51fe59c640 1 --2- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f51e0078040 0x7f51e007a500 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51f0033000 tx=0x7f51f0004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7f4ac89a8640 1 --2- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4aa8077f70 0x7f4aa807a430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8000c00 tx=0x7f4ab800b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7ffa177fe640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa1c077fa0 0x7ffa1c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7ffa177fe640 1 -- 172.21.15.98:0/2662901720 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffa240b9e60 con 0x7ffa301f8a10 2024-09-10T20:59:09.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7ffa177fe640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.98:6808/2541446963,v1:172.21.15.98:6809/2541446963] conn(0x7ffa1c081720 0x7ffa1c083b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7ffa177fe640 1 -- 172.21.15.98:0/2662901720 --> [v2:172.21.15.98:6808/2541446963,v1:172.21.15.98:6809/2541446963] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa24036e70 con 0x7ffa1c081720 2024-09-10T20:59:09.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7ffa177fe640 1 -- 172.21.15.98:0/2662901720 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa240ba250 con 0x7ffa301f8a10 2024-09-10T20:59:09.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7f9feeb52640 1 -- 172.21.15.98:0/1432281821 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9fe80cfa60 con 0x7f9fe80cdb10 2024-09-10T20:59:09.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7f51fed9d640 1 --2- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:6826/2783440087,v1:172.21.15.190:6827/2783440087] conn(0x7f51e00817b0 0x7f51e0083bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7ffa2f7fe640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.98:6808/2541446963,v1:172.21.15.98:6809/2541446963] conn(0x7ffa1c081720 0x7ffa1c083b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7ffa2effd640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa1c077fa0 0x7ffa1c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7f598a7fc640 1 -- 172.21.15.98:0/1301720631 <== osd.0 v2:172.21.15.98:6824/3415066506 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5984013e70 con 0x7f59780816d0 2024-09-10T20:59:09.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.069+0000 7f4ac17fa640 1 -- 172.21.15.98:0/2743062534 <== osd.7 v2:172.21.15.190:6818/3616946293 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ab001fe50 con 0x7f4aa8081670 2024-09-10T20:59:09.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.068+0000 7f356bfff640 1 --2- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3564078040 0x7f356407a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.069+0000 7ffa2f7fe640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.98:6808/2541446963,v1:172.21.15.98:6809/2541446963] conn(0x7ffa1c081720 0x7ffa1c083b60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.069+0000 7f356bfff640 1 -- 172.21.15.98:0/1559210875 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f35740bcb60 con 0x7f3578005a60 2024-09-10T20:59:09.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.069+0000 7f2e41ffb640 1 -- 172.21.15.98:0/3896199643 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2e44004b40 con 0x7f2e4c15f0f0 2024-09-10T20:59:09.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.069+0000 7f356bfff640 1 --2- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:6802/904517515,v1:172.21.15.190:6803/904517515] conn(0x7f35640817c0 0x7f3564083c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.069+0000 7f51eeffd640 1 -- 172.21.15.98:0/159858533 <== osd.4 v2:172.21.15.190:6826/2783440087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f51e00842c0 con 0x7f51e00817b0 2024-09-10T20:59:09.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.069+0000 7f63b97fa640 1 -- 172.21.15.98:0/275509787 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f63ac024060 con 0x7f63b40366f0 2024-09-10T20:59:09.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.069+0000 7f356bfff640 1 -- 172.21.15.98:0/1559210875 --> [v2:172.21.15.190:6802/904517515,v1:172.21.15.190:6803/904517515] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35640842d0 con 0x7f35640817c0 2024-09-10T20:59:09.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.069+0000 7f356bfff640 1 -- 172.21.15.98:0/1559210875 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f35740bcf50 con 0x7f3578005a60 2024-09-10T20:59:09.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.069+0000 7f357effd640 1 --2- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3564078040 0x7f356407a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f2e41ffb640 1 --2- 172.21.15.98:0/3896199643 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2e30078040 0x7f2e3007a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7ffa2effd640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa1c077fa0 0x7ffa1c07a460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffa300cc160 tx=0x7ffa20005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f2e41ffb640 1 -- 172.21.15.98:0/3896199643 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2e4409f170 con 0x7f2e4c15f0f0 2024-09-10T20:59:09.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f63b97fa640 1 --2- 172.21.15.98:0/275509787 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f63a8077f70 0x7f63a807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f2e41ffb640 1 --2- 172.21.15.98:0/3896199643 >> [v2:172.21.15.98:6816/2538250805,v1:172.21.15.98:6817/2538250805] conn(0x7f2e300817c0 0x7f2e30083c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f357effd640 1 --2- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3564078040 0x7f356407a500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3578047be0 tx=0x7f356c005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f63b97fa640 1 -- 172.21.15.98:0/275509787 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f63ac09a9d0 con 0x7f63b40366f0 2024-09-10T20:59:09.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7ffa177fe640 1 -- 172.21.15.98:0/2662901720 <== osd.2 v2:172.21.15.98:6808/2541446963 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa24036e70 con 0x7ffa1c081720 2024-09-10T20:59:09.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f2e41ffb640 1 -- 172.21.15.98:0/3896199643 --> [v2:172.21.15.98:6816/2538250805,v1:172.21.15.98:6817/2538250805] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e44013c60 con 0x7f2e300817c0 2024-09-10T20:59:09.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f2e41ffb640 1 -- 172.21.15.98:0/3896199643 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2e4409f560 con 0x7f2e4c15f0f0 2024-09-10T20:59:09.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f357ffff640 1 --2- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:6802/904517515,v1:172.21.15.190:6803/904517515] conn(0x7f35640817c0 0x7f3564083c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f63b97fa640 1 --2- 172.21.15.98:0/275509787 >> [v2:172.21.15.98:6800/3091760272,v1:172.21.15.98:6801/3091760272] conn(0x7f63a80816e0 0x7f63a8083b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f9fe57fa640 1 -- 172.21.15.98:0/1432281821 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9fe0018420 con 0x7f9fe80cdb10 2024-09-10T20:59:09.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f2e516ee640 1 --2- 172.21.15.98:0/3896199643 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2e30078040 0x7f2e3007a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f63b97fa640 1 -- 172.21.15.98:0/275509787 --> [v2:172.21.15.98:6800/3091760272,v1:172.21.15.98:6801/3091760272] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63ac026e50 con 0x7f63a80816e0 2024-09-10T20:59:09.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.071+0000 7f63b97fa640 1 -- 172.21.15.98:0/275509787 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f63ac09adc0 con 0x7f63b40366f0 2024-09-10T20:59:09.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.071+0000 7f63bbfff640 1 --2- 172.21.15.98:0/275509787 >> [v2:172.21.15.98:6800/3091760272,v1:172.21.15.98:6801/3091760272] conn(0x7f63a80816e0 0x7f63a8083b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.071+0000 7f9fe57fa640 1 --2- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9fd4078040 0x7f9fd407a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f2e50eed640 1 --2- 172.21.15.98:0/3896199643 >> [v2:172.21.15.98:6816/2538250805,v1:172.21.15.98:6817/2538250805] conn(0x7f2e300817c0 0x7f2e30083c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.071+0000 7f9fe57fa640 1 -- 172.21.15.98:0/1432281821 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9fe009af40 con 0x7f9fe80cdb10 2024-09-10T20:59:09.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.071+0000 7f9fe7fff640 1 --2- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9fd4078040 0x7f9fd407a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.071+0000 7f9fe57fa640 1 --2- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:6810/2106840781,v1:172.21.15.190:6811/2106840781] conn(0x7f9fd40817b0 0x7f9fd4083bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.071+0000 7f9fe57fa640 1 -- 172.21.15.98:0/1432281821 --> [v2:172.21.15.190:6810/2106840781,v1:172.21.15.190:6811/2106840781] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9fe001ae50 con 0x7f9fd40817b0 2024-09-10T20:59:09.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.071+0000 7f9fe57fa640 1 -- 172.21.15.98:0/1432281821 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9fe009d5a0 con 0x7f9fe80cdb10 2024-09-10T20:59:09.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.071+0000 7f63bbfff640 1 --2- 172.21.15.98:0/275509787 >> [v2:172.21.15.98:6800/3091760272,v1:172.21.15.98:6801/3091760272] conn(0x7f63a80816e0 0x7f63a8083b20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.071+0000 7f9fed0ef640 1 --2- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:6810/2106840781,v1:172.21.15.190:6811/2106840781] conn(0x7f9fd40817b0 0x7f9fd4083bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.072+0000 7f2e50eed640 1 --2- 172.21.15.98:0/3896199643 >> [v2:172.21.15.98:6816/2538250805,v1:172.21.15.98:6817/2538250805] conn(0x7f2e300817c0 0x7f2e30083c00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.072+0000 7f2e516ee640 1 --2- 172.21.15.98:0/3896199643 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2e30078040 0x7f2e3007a500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2e480095f0 tx=0x7f2e48004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.070+0000 7f63c0f97640 1 --2- 172.21.15.98:0/275509787 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f63a8077f70 0x7f63a807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.072+0000 7f9fed0ef640 1 --2- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:6810/2106840781,v1:172.21.15.190:6811/2106840781] conn(0x7f9fd40817b0 0x7f9fd4083bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.072+0000 7f9fe7fff640 1 --2- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9fd4078040 0x7f9fd407a500 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc033000 tx=0x7f9fdc0046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.072+0000 7f63b97fa640 1 -- 172.21.15.98:0/275509787 <== osd.1 v2:172.21.15.98:6800/3091760272 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f63ac026e50 con 0x7f63a80816e0 2024-09-10T20:59:09.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.071+0000 7f357ffff640 1 --2- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:6802/904517515,v1:172.21.15.190:6803/904517515] conn(0x7f35640817c0 0x7f3564083c00 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.072+0000 7f356bfff640 1 -- 172.21.15.98:0/1559210875 <== osd.5 v2:172.21.15.190:6802/904517515 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35640842d0 con 0x7f35640817c0 2024-09-10T20:59:09.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.073+0000 7f9fe57fa640 1 -- 172.21.15.98:0/1432281821 <== osd.6 v2:172.21.15.190:6810/2106840781 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9fe001ae50 con 0x7f9fd40817b0 2024-09-10T20:59:09.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.074+0000 7f63c0f97640 1 --2- 172.21.15.98:0/275509787 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f63a8077f70 0x7f63a807a430 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f63b40550e0 tx=0x7f63bc06a670 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.074+0000 7f2e41ffb640 1 -- 172.21.15.98:0/3896199643 <== osd.3 v2:172.21.15.98:6816/2538250805 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e44013c60 con 0x7f2e300817c0 2024-09-10T20:59:09.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.099+0000 7f4acac0c640 1 -- 172.21.15.98:0/2743062534 --> [v2:172.21.15.190:6818/3616946293,v1:172.21.15.190:6819/3616946293] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ac41609e0 con 0x7f4aa8081670 2024-09-10T20:59:09.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.099+0000 7f51ff59e640 1 -- 172.21.15.98:0/159858533 --> [v2:172.21.15.190:6826/2783440087,v1:172.21.15.190:6827/2783440087] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f51e4002dd0 con 0x7f51e00817b0 2024-09-10T20:59:09.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.100+0000 7f51eeffd640 1 -- 172.21.15.98:0/159858533 <== osd.4 v2:172.21.15.190:6826/2783440087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f51e4002dd0 con 0x7f51e00817b0 2024-09-10T20:59:09.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.100+0000 7f4ac17fa640 1 -- 172.21.15.98:0/2743062534 <== osd.7 v2:172.21.15.190:6818/3616946293 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4ac41609e0 con 0x7f4aa8081670 2024-09-10T20:59:09.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.100+0000 7f599c175640 1 -- 172.21.15.98:0/1301720631 --> [v2:172.21.15.98:6824/3415066506,v1:172.21.15.98:6825/3415066506] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f597c002d10 con 0x7f59780816d0 2024-09-10T20:59:09.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.100+0000 7f51ff59e640 1 -- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:6826/2783440087,v1:172.21.15.190:6827/2783440087] conn(0x7f51e00817b0 msgr2=0x7f51e0083bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.100+0000 7f51ff59e640 1 --2- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:6826/2783440087,v1:172.21.15.190:6827/2783440087] conn(0x7f51e00817b0 0x7f51e0083bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.100+0000 7ffa35452640 1 -- 172.21.15.98:0/2662901720 --> [v2:172.21.15.98:6808/2541446963,v1:172.21.15.98:6809/2541446963] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa300d0080 con 0x7ffa1c081720 2024-09-10T20:59:09.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f4aaeffd640 1 -- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:6818/3616946293,v1:172.21.15.190:6819/3616946293] conn(0x7f4aa8081670 msgr2=0x7f4aa8083ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f4aaeffd640 1 --2- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:6818/3616946293,v1:172.21.15.190:6819/3616946293] conn(0x7f4aa8081670 0x7f4aa8083ab0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f51ff59e640 1 -- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f51e0078040 msgr2=0x7f51e007a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f51ff59e640 1 --2- 172.21.15.98:0/159858533 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f51e0078040 0x7f51e007a500 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51f0033000 tx=0x7f51f0004730 comp rx=0 tx=0).stop 2024-09-10T20:59:09.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f4aaeffd640 1 -- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4aa8077f70 msgr2=0x7f4aa807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f4aaeffd640 1 --2- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4aa8077f70 0x7f4aa807a430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8000c00 tx=0x7f4ab800b040 comp rx=0 tx=0).stop 2024-09-10T20:59:09.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f4aaeffd640 1 -- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac40d01b0 msgr2=0x7f4ac4151780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f4aaeffd640 1 --2- 172.21.15.98:0/2743062534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac40d01b0 0x7f4ac4151780 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000c3c0 tx=0x7f4ab000c880 comp rx=0 tx=0).stop 2024-09-10T20:59:09.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f51ff59e640 1 -- 172.21.15.98:0/159858533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f8034ba0 msgr2=0x7f51f800c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f51ff59e640 1 --2- 172.21.15.98:0/159858533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f8034ba0 0x7f51f800c110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f520006a990 tx=0x7f520006adf0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f4ac91a9640 1 -- 172.21.15.98:0/2743062534 reap_dead start 2024-09-10T20:59:09.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f51fed9d640 1 -- 172.21.15.98:0/159858533 reap_dead start 2024-09-10T20:59:09.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f4aaeffd640 1 -- 172.21.15.98:0/2743062534 shutdown_connections 2024-09-10T20:59:09.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f4aaeffd640 1 -- 172.21.15.98:0/2743062534 >> 172.21.15.98:0/2743062534 conn(0x7f4ac409d170 msgr2=0x7f4ac415f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f51ff59e640 1 -- 172.21.15.98:0/159858533 shutdown_connections 2024-09-10T20:59:09.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f51ff59e640 1 -- 172.21.15.98:0/159858533 >> 172.21.15.98:0/159858533 conn(0x7f51f801e890 msgr2=0x7f51f80509e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7ffa177fe640 1 -- 172.21.15.98:0/2662901720 <== osd.2 v2:172.21.15.98:6808/2541446963 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffa300d0080 con 0x7ffa1c081720 2024-09-10T20:59:09.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f598a7fc640 1 -- 172.21.15.98:0/1301720631 <== osd.0 v2:172.21.15.98:6824/3415066506 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f597c002d10 con 0x7f59780816d0 2024-09-10T20:59:09.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f51ff59e640 1 -- 172.21.15.98:0/159858533 shutdown_connections 2024-09-10T20:59:09.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7f51ff59e640 1 -- 172.21.15.98:0/159858533 wait complete. 2024-09-10T20:59:09.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7ffa35452640 1 -- 172.21.15.98:0/2662901720 >> [v2:172.21.15.98:6808/2541446963,v1:172.21.15.98:6809/2541446963] conn(0x7ffa1c081720 msgr2=0x7ffa1c083b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.101+0000 7ffa35452640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.98:6808/2541446963,v1:172.21.15.98:6809/2541446963] conn(0x7ffa1c081720 0x7ffa1c083b60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7ffa35452640 1 -- 172.21.15.98:0/2662901720 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa1c077fa0 msgr2=0x7ffa1c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7ffa35452640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa1c077fa0 0x7ffa1c07a460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffa300cc160 tx=0x7ffa20005f90 comp rx=0 tx=0).stop 2024-09-10T20:59:09.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7ffa35452640 1 -- 172.21.15.98:0/2662901720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa301f8a10 msgr2=0x7ffa300acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7ffa35452640 1 --2- 172.21.15.98:0/2662901720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa301f8a10 0x7ffa300acb50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffa24032000 tx=0x7ffa24033670 comp rx=0 tx=0).stop 2024-09-10T20:59:09.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7ffa2f7fe640 1 -- 172.21.15.98:0/2662901720 reap_dead start 2024-09-10T20:59:09.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7f4aaeffd640 1 -- 172.21.15.98:0/2743062534 shutdown_connections 2024-09-10T20:59:09.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7f4aaeffd640 1 -- 172.21.15.98:0/2743062534 wait complete. 2024-09-10T20:59:09.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7ffa35452640 1 -- 172.21.15.98:0/2662901720 shutdown_connections 2024-09-10T20:59:09.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7ffa35452640 1 -- 172.21.15.98:0/2662901720 >> 172.21.15.98:0/2662901720 conn(0x7ffa300cfaf0 msgr2=0x7ffa300b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7ffa35452640 1 -- 172.21.15.98:0/2662901720 shutdown_connections 2024-09-10T20:59:09.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7ffa35452640 1 -- 172.21.15.98:0/2662901720 wait complete. 2024-09-10T20:59:09.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7f599c175640 1 -- 172.21.15.98:0/1301720631 >> [v2:172.21.15.98:6824/3415066506,v1:172.21.15.98:6825/3415066506] conn(0x7f59780816d0 msgr2=0x7f5978083b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7f599c175640 1 --2- 172.21.15.98:0/1301720631 >> [v2:172.21.15.98:6824/3415066506,v1:172.21.15.98:6825/3415066506] conn(0x7f59780816d0 0x7f5978083b10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7f599c175640 1 -- 172.21.15.98:0/1301720631 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5978077f50 msgr2=0x7f597807a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7f599c175640 1 --2- 172.21.15.98:0/1301720631 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5978077f50 0x7f597807a410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5990035000 tx=0x7f59900046c0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7f599c175640 1 -- 172.21.15.98:0/1301720631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59940c8510 msgr2=0x7f59940c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.102+0000 7f599c175640 1 --2- 172.21.15.98:0/1301720631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59940c8510 0x7f59940c6da0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f598400c500 tx=0x7f598400c9c0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.103+0000 7f9feeb52640 1 -- 172.21.15.98:0/1432281821 --> [v2:172.21.15.190:6810/2106840781,v1:172.21.15.190:6811/2106840781] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9fe815c680 con 0x7f9fd40817b0 2024-09-10T20:59:09.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.103+0000 7f599a712640 1 -- 172.21.15.98:0/1301720631 reap_dead start 2024-09-10T20:59:09.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.103+0000 7f599c175640 1 -- 172.21.15.98:0/1301720631 shutdown_connections 2024-09-10T20:59:09.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.103+0000 7f599c175640 1 -- 172.21.15.98:0/1301720631 >> 172.21.15.98:0/1301720631 conn(0x7f59940d1d10 msgr2=0x7f599406e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.103+0000 7f9fe57fa640 1 -- 172.21.15.98:0/1432281821 <== osd.6 v2:172.21.15.190:6810/2106840781 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9fe815c680 con 0x7f9fd40817b0 2024-09-10T20:59:09.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.103+0000 7f599c175640 1 -- 172.21.15.98:0/1301720631 shutdown_connections 2024-09-10T20:59:09.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.103+0000 7f599c175640 1 -- 172.21.15.98:0/1301720631 wait complete. 2024-09-10T20:59:09.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.104+0000 7f2e53952640 1 -- 172.21.15.98:0/3896199643 --> [v2:172.21.15.98:6816/2538250805,v1:172.21.15.98:6817/2538250805] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2e2c002dd0 con 0x7f2e300817c0 2024-09-10T20:59:09.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.104+0000 7f3584fc8640 1 -- 172.21.15.98:0/1559210875 --> [v2:172.21.15.190:6802/904517515,v1:172.21.15.190:6803/904517515] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f35780078b0 con 0x7f35640817c0 2024-09-10T20:59:09.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.104+0000 7f9feeb52640 1 -- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:6810/2106840781,v1:172.21.15.190:6811/2106840781] conn(0x7f9fd40817b0 msgr2=0x7f9fd4083bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.104+0000 7f9feeb52640 1 --2- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:6810/2106840781,v1:172.21.15.190:6811/2106840781] conn(0x7f9fd40817b0 0x7f9fd4083bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.104+0000 7f63a2ffd640 1 -- 172.21.15.98:0/275509787 --> [v2:172.21.15.98:6800/3091760272,v1:172.21.15.98:6801/3091760272] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f63b4049d70 con 0x7f63a80816e0 2024-09-10T20:59:09.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.104+0000 7f9feeb52640 1 -- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9fd4078040 msgr2=0x7f9fd407a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.104+0000 7f9feeb52640 1 --2- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9fd4078040 0x7f9fd407a500 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc033000 tx=0x7f9fdc0046c0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.104+0000 7f2e41ffb640 1 -- 172.21.15.98:0/3896199643 <== osd.3 v2:172.21.15.98:6816/2538250805 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2e2c002dd0 con 0x7f2e300817c0 2024-09-10T20:59:09.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.104+0000 7f9feeb52640 1 -- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fe80cdb10 msgr2=0x7f9fe81f8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.104+0000 7f9feeb52640 1 --2- 172.21.15.98:0/1432281821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fe80cdb10 0x7f9fe81f8c20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000ebc0 tx=0x7f9fe000c6a0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.104+0000 7f9fed0ef640 1 -- 172.21.15.98:0/1432281821 reap_dead start 2024-09-10T20:59:09.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.104+0000 7f9feeb52640 1 -- 172.21.15.98:0/1432281821 shutdown_connections 2024-09-10T20:59:09.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.104+0000 7f9feeb52640 1 -- 172.21.15.98:0/1432281821 >> 172.21.15.98:0/1432281821 conn(0x7f9fe806e480 msgr2=0x7f9fe80d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f356bfff640 1 -- 172.21.15.98:0/1559210875 <== osd.5 v2:172.21.15.190:6802/904517515 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f35780078b0 con 0x7f35640817c0 2024-09-10T20:59:09.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f63b97fa640 1 -- 172.21.15.98:0/275509787 <== osd.1 v2:172.21.15.98:6800/3091760272 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f63b4049d70 con 0x7f63a80816e0 2024-09-10T20:59:09.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f2e53952640 1 -- 172.21.15.98:0/3896199643 >> [v2:172.21.15.98:6816/2538250805,v1:172.21.15.98:6817/2538250805] conn(0x7f2e300817c0 msgr2=0x7f2e30083c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f2e53952640 1 --2- 172.21.15.98:0/3896199643 >> [v2:172.21.15.98:6816/2538250805,v1:172.21.15.98:6817/2538250805] conn(0x7f2e300817c0 0x7f2e30083c00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f2e53952640 1 -- 172.21.15.98:0/3896199643 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2e30078040 msgr2=0x7f2e3007a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f2e53952640 1 --2- 172.21.15.98:0/3896199643 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2e30078040 0x7f2e3007a500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2e480095f0 tx=0x7f2e48004730 comp rx=0 tx=0).stop 2024-09-10T20:59:09.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f2e53952640 1 -- 172.21.15.98:0/3896199643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c15f0f0 msgr2=0x7f2e4c1521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f2e53952640 1 --2- 172.21.15.98:0/3896199643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e4c15f0f0 0x7f2e4c1521b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2e44005fd0 tx=0x7f2e4400c6a0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f9feeb52640 1 -- 172.21.15.98:0/1432281821 shutdown_connections 2024-09-10T20:59:09.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f9feeb52640 1 -- 172.21.15.98:0/1432281821 wait complete. 2024-09-10T20:59:09.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f2e51eef640 1 -- 172.21.15.98:0/3896199643 reap_dead start 2024-09-10T20:59:09.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f3584fc8640 1 -- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:6802/904517515,v1:172.21.15.190:6803/904517515] conn(0x7f35640817c0 msgr2=0x7f3564083c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f3584fc8640 1 --2- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:6802/904517515,v1:172.21.15.190:6803/904517515] conn(0x7f35640817c0 0x7f3564083c00 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f3584fc8640 1 -- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3564078040 msgr2=0x7f356407a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f3584fc8640 1 --2- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3564078040 0x7f356407a500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3578047be0 tx=0x7f356c005f90 comp rx=0 tx=0).stop 2024-09-10T20:59:09.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f3584fc8640 1 -- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3578005a60 msgr2=0x7f3578049da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f3584fc8640 1 --2- 172.21.15.98:0/1559210875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3578005a60 0x7f3578049da0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3574009470 tx=0x7f3574005270 comp rx=0 tx=0).stop 2024-09-10T20:59:09.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.106+0000 7f357ffff640 1 -- 172.21.15.98:0/1559210875 reap_dead start 2024-09-10T20:59:09.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.106+0000 7f3584fc8640 1 -- 172.21.15.98:0/1559210875 shutdown_connections 2024-09-10T20:59:09.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.106+0000 7f3584fc8640 1 -- 172.21.15.98:0/1559210875 >> 172.21.15.98:0/1559210875 conn(0x7f3578054560 msgr2=0x7f3578052a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f2e53952640 1 -- 172.21.15.98:0/3896199643 shutdown_connections 2024-09-10T20:59:09.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.106+0000 7f3584fc8640 1 -- 172.21.15.98:0/1559210875 shutdown_connections 2024-09-10T20:59:09.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.106+0000 7f3584fc8640 1 -- 172.21.15.98:0/1559210875 wait complete. 2024-09-10T20:59:09.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.106+0000 7f63a2ffd640 1 -- 172.21.15.98:0/275509787 >> [v2:172.21.15.98:6800/3091760272,v1:172.21.15.98:6801/3091760272] conn(0x7f63a80816e0 msgr2=0x7f63a8083b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.106+0000 7f63a2ffd640 1 --2- 172.21.15.98:0/275509787 >> [v2:172.21.15.98:6800/3091760272,v1:172.21.15.98:6801/3091760272] conn(0x7f63a80816e0 0x7f63a8083b20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.106+0000 7f63a2ffd640 1 -- 172.21.15.98:0/275509787 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f63a8077f70 msgr2=0x7f63a807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.106+0000 7f63a2ffd640 1 --2- 172.21.15.98:0/275509787 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f63a8077f70 0x7f63a807a430 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f63b40550e0 tx=0x7f63bc06a670 comp rx=0 tx=0).stop 2024-09-10T20:59:09.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.106+0000 7f63a2ffd640 1 -- 172.21.15.98:0/275509787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40366f0 msgr2=0x7f63b40d5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.106+0000 7f63a2ffd640 1 --2- 172.21.15.98:0/275509787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b40366f0 0x7f63b40d5ac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f63ac007860 tx=0x7f63ac007d20 comp rx=0 tx=0).stop 2024-09-10T20:59:09.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.105+0000 7f2e53952640 1 -- 172.21.15.98:0/3896199643 >> 172.21.15.98:0/3896199643 conn(0x7f2e4c06e480 msgr2=0x7f2e4c0cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.106+0000 7f2e53952640 1 -- 172.21.15.98:0/3896199643 shutdown_connections 2024-09-10T20:59:09.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.106+0000 7f2e53952640 1 -- 172.21.15.98:0/3896199643 wait complete. 2024-09-10T20:59:09.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.106+0000 7f63c1798640 1 -- 172.21.15.98:0/275509787 reap_dead start 2024-09-10T20:59:09.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.107+0000 7f63a2ffd640 1 -- 172.21.15.98:0/275509787 shutdown_connections 2024-09-10T20:59:09.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.107+0000 7f63a2ffd640 1 -- 172.21.15.98:0/275509787 >> 172.21.15.98:0/275509787 conn(0x7f63b4054560 msgr2=0x7f63b4053600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.107+0000 7f63a2ffd640 1 -- 172.21.15.98:0/275509787 shutdown_connections 2024-09-10T20:59:09.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.107+0000 7f63a2ffd640 1 -- 172.21.15.98:0/275509787 wait complete. 2024-09-10T20:59:09.151 INFO:teuthology.orchestra.run.smithi098.stdout:64424509450 2024-09-10T20:59:09.151 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T20:59:09.152 INFO:teuthology.orchestra.run.smithi098.stdout:60129542153 2024-09-10T20:59:09.152 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T20:59:09.153 INFO:teuthology.orchestra.run.smithi098.stdout:64424509450 2024-09-10T20:59:09.153 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T20:59:09.154 INFO:teuthology.orchestra.run.smithi098.stdout:60129542153 2024-09-10T20:59:09.154 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:59:09.155 INFO:teuthology.orchestra.run.smithi098.stdout:60129542153 2024-09-10T20:59:09.155 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T20:59:09.156 INFO:teuthology.orchestra.run.smithi098.stdout:55834574858 2024-09-10T20:59:09.156 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T20:59:09.157 INFO:teuthology.orchestra.run.smithi098.stdout:64424509450 2024-09-10T20:59:09.157 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T20:59:09.158 INFO:teuthology.orchestra.run.smithi098.stdout:60129542153 2024-09-10T20:59:09.158 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T20:59:09.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.361+0000 7f3614f13640 1 -- 172.21.15.98:0/778528246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3608034650 msgr2=0x7f3608050240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.361+0000 7f3614f13640 1 --2- 172.21.15.98:0/778528246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3608034650 0x7f3608050240 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3604005fd0 tx=0x7f360402a4b0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.361+0000 7f3614f13640 1 -- 172.21.15.98:0/778528246 shutdown_connections 2024-09-10T20:59:09.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.361+0000 7f3614f13640 1 --2- 172.21.15.98:0/778528246 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3608034650 0x7f3608050240 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.361+0000 7f3614f13640 1 --2- 172.21.15.98:0/778528246 >> v2:172.21.15.98:3300/0 conn(0x7f3608034ba0 0x7f360804fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.361+0000 7f3614f13640 1 -- 172.21.15.98:0/778528246 >> 172.21.15.98:0/778528246 conn(0x7f360801e890 msgr2=0x7f3608055640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.361+0000 7f3614f13640 1 -- 172.21.15.98:0/778528246 shutdown_connections 2024-09-10T20:59:09.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.363+0000 7f54f0807640 1 -- 172.21.15.98:0/3024847262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54ec0704a0 msgr2=0x7f54ec0741b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.363+0000 7f54f0807640 1 --2- 172.21.15.98:0/3024847262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54ec0704a0 0x7f54ec0741b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0b4ba0 tx=0x7f54d402a480 comp rx=0 tx=0).stop 2024-09-10T20:59:09.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.363+0000 7f54f0807640 1 -- 172.21.15.98:0/3024847262 shutdown_connections 2024-09-10T20:59:09.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.363+0000 7f54f0807640 1 --2- 172.21.15.98:0/3024847262 >> v2:172.21.15.190:3300/0 conn(0x7f54ec15d5e0 0x7f54ec15da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.363+0000 7f54f0807640 1 --2- 172.21.15.98:0/3024847262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54ec0704a0 0x7f54ec0741b0 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0b4ba0 tx=0x7f54d402a480 comp rx=0 tx=0).stop 2024-09-10T20:59:09.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.363+0000 7f54f0807640 1 -- 172.21.15.98:0/3024847262 >> 172.21.15.98:0/3024847262 conn(0x7f54ec06ce40 msgr2=0x7f54ec06d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.361+0000 7f3614f13640 1 -- 172.21.15.98:0/778528246 wait complete. 2024-09-10T20:59:09.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.363+0000 7f3614f13640 1 Processor -- start 2024-09-10T20:59:09.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.364+0000 7f54f0807640 1 -- 172.21.15.98:0/3024847262 shutdown_connections 2024-09-10T20:59:09.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.364+0000 7f3614f13640 1 -- start start 2024-09-10T20:59:09.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.364+0000 7f54f0807640 1 -- 172.21.15.98:0/3024847262 wait complete. 2024-09-10T20:59:09.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.365+0000 7f00c1423640 1 -- 172.21.15.98:0/3215742892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00b4037ee0 msgr2=0x7f00b40e6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.365+0000 7f00c1423640 1 --2- 172.21.15.98:0/3215742892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00b4037ee0 0x7f00b40e6940 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f00b400f290 tx=0x7f00a802a5e0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.365+0000 7f00c1423640 1 -- 172.21.15.98:0/3215742892 shutdown_connections 2024-09-10T20:59:09.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.365+0000 7f00c1423640 1 --2- 172.21.15.98:0/3215742892 >> v2:172.21.15.98:3300/0 conn(0x7f00b403b910 0x7f00b40066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.365+0000 7f00c1423640 1 --2- 172.21.15.98:0/3215742892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00b4037ee0 0x7f00b40e6940 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.365+0000 7f00c1423640 1 --2- 172.21.15.98:0/3215742892 >> v2:172.21.15.190:3300/0 conn(0x7f00b40e2770 0x7f00b40e6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.365+0000 7f00c1423640 1 -- 172.21.15.98:0/3215742892 >> 172.21.15.98:0/3215742892 conn(0x7f00b403a3a0 msgr2=0x7f00b403a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.365+0000 7f3614f13640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3608008d90 0x7f360817ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.365+0000 7f00c1423640 1 -- 172.21.15.98:0/3215742892 shutdown_connections 2024-09-10T20:59:09.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.365+0000 7f00c1423640 1 -- 172.21.15.98:0/3215742892 wait complete. 2024-09-10T20:59:09.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.365+0000 7f360f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3608008d90 0x7f360817ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.365+0000 7f360f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3608008d90 0x7f360817ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58696/0 (socket says 172.21.15.98:58696) 2024-09-10T20:59:09.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f6754352640 1 -- 172.21.15.98:0/3138699570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6738004800 msgr2=0x7f6738004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f360f7fe640 1 -- 172.21.15.98:0/3437500420 learned_addr learned my addr 172.21.15.98:0/3437500420 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f6754352640 1 --2- 172.21.15.98:0/3138699570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6738004800 0x7f6738004c80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f673c009b30 tx=0x7f6738005830 comp rx=0 tx=0).stop 2024-09-10T20:59:09.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f6754352640 1 -- 172.21.15.98:0/3138699570 shutdown_connections 2024-09-10T20:59:09.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f3da63a3640 1 -- 172.21.15.98:0/3652928265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d8c003a20 msgr2=0x7f3d8c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f3da63a3640 1 --2- 172.21.15.98:0/3652928265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d8c003a20 0x7f3d8c003e40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009b50 tx=0x7f3d8c005830 comp rx=0 tx=0).stop 2024-09-10T20:59:09.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f3da4940640 1 -- 172.21.15.98:0/3652928265 reap_dead start 2024-09-10T20:59:09.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f6754352640 1 --2- 172.21.15.98:0/3138699570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6738004800 0x7f6738004c80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f6754352640 1 -- 172.21.15.98:0/3138699570 >> 172.21.15.98:0/3138699570 conn(0x7f674401e890 msgr2=0x7f6744055640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f3614f13640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3608034650 0x7f36080dd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f54f0807640 1 Processor -- start 2024-09-10T20:59:09.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f3614f13640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3608034ba0 0x7f3608051f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f3614f13640 1 -- 172.21.15.98:0/3437500420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3608055080 con 0x7f3608008d90 2024-09-10T20:59:09.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f6754352640 1 -- 172.21.15.98:0/3138699570 shutdown_connections 2024-09-10T20:59:09.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f3614f13640 1 -- 172.21.15.98:0/3437500420 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3608050780 con 0x7f3608034ba0 2024-09-10T20:59:09.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f3614f13640 1 -- 172.21.15.98:0/3437500420 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f36080dbce0 con 0x7f3608034650 2024-09-10T20:59:09.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f54f0807640 1 -- start start 2024-09-10T20:59:09.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f6754352640 1 -- 172.21.15.98:0/3138699570 wait complete. 2024-09-10T20:59:09.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f360ffff640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3608034ba0 0x7f3608051f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f360effd640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3608034650 0x7f36080dd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f360f7fe640 1 -- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3608034650 msgr2=0x7f36080dd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f360f7fe640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3608034650 0x7f36080dd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f360f7fe640 1 -- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3608034ba0 msgr2=0x7f3608051f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f360f7fe640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3608034ba0 0x7f3608051f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f360f7fe640 1 -- 172.21.15.98:0/3437500420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36081870c0 con 0x7f3608008d90 2024-09-10T20:59:09.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f360f7fe640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3608008d90 0x7f360817ca60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00c780 tx=0x7f35fc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f7e6b584640 1 -- 172.21.15.98:0/3960300115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e64159b80 msgr2=0x7f7e6415e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f7e6b584640 1 --2- 172.21.15.98:0/3960300115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e64159b80 0x7f7e6415e140 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7e600097b0 tx=0x7f7e6002a4f0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f7e6b584640 1 -- 172.21.15.98:0/3960300115 shutdown_connections 2024-09-10T20:59:09.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f7e6b584640 1 --2- 172.21.15.98:0/3960300115 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e64159b80 0x7f7e6415e140 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f7e6b584640 1 -- 172.21.15.98:0/3960300115 >> 172.21.15.98:0/3960300115 conn(0x7f7e6406ca50 msgr2=0x7f7e6406ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f0e6965b640 1 -- 172.21.15.98:0/2114102466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e50003aa0 msgr2=0x7f0e50003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f0e6965b640 1 --2- 172.21.15.98:0/2114102466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e50003aa0 0x7f0e50003ec0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c530 tx=0x7f0e50005830 comp rx=0 tx=0).stop 2024-09-10T20:59:09.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f0e6965b640 1 -- 172.21.15.98:0/2114102466 shutdown_connections 2024-09-10T20:59:09.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f0e6965b640 1 --2- 172.21.15.98:0/2114102466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e50003aa0 0x7f0e50003ec0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f0e6965b640 1 -- 172.21.15.98:0/2114102466 >> 172.21.15.98:0/2114102466 conn(0x7f0e6406ce40 msgr2=0x7f0e6406d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f0e6965b640 1 -- 172.21.15.98:0/2114102466 shutdown_connections 2024-09-10T20:59:09.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f3da63a3640 1 -- 172.21.15.98:0/3652928265 shutdown_connections 2024-09-10T20:59:09.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f3da63a3640 1 -- 172.21.15.98:0/3652928265 >> 172.21.15.98:0/3652928265 conn(0x7f3da00d0940 msgr2=0x7f3da00d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f3da63a3640 1 -- 172.21.15.98:0/3652928265 shutdown_connections 2024-09-10T20:59:09.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.366+0000 7f3da63a3640 1 -- 172.21.15.98:0/3652928265 wait complete. 2024-09-10T20:59:09.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f00c1423640 1 Processor -- start 2024-09-10T20:59:09.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.368+0000 7f00c1423640 1 -- start start 2024-09-10T20:59:09.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.368+0000 7f35fbfff640 1 -- 172.21.15.98:0/3437500420 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc016020 con 0x7f3608008d90 2024-09-10T20:59:09.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.368+0000 7f3614f13640 1 -- 172.21.15.98:0/3437500420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36080dac40 con 0x7f3608008d90 2024-09-10T20:59:09.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.368+0000 7f7e6b584640 1 -- 172.21.15.98:0/3960300115 shutdown_connections 2024-09-10T20:59:09.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.368+0000 7f54f0807640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54ec0b29f0 0x7f54ec1474c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.368+0000 7f6754352640 1 Processor -- start 2024-09-10T20:59:09.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.368+0000 7f7e6b584640 1 -- 172.21.15.98:0/3960300115 wait complete. 2024-09-10T20:59:09.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54f0807640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54ec15d5e0 0x7f54ec166a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54f0807640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54ec14a170 0x7f54ec168ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54ead9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54ec0b29f0 0x7f54ec1474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54ead9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54ec0b29f0 0x7f54ec1474c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:35234/0 (socket says 172.21.15.98:35234) 2024-09-10T20:59:09.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54ead9d640 1 -- 172.21.15.98:0/941072153 learned_addr learned my addr 172.21.15.98:0/941072153 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54f0807640 1 -- 172.21.15.98:0/941072153 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec06e8d0 con 0x7f54ec14a170 2024-09-10T20:59:09.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54f0807640 1 -- 172.21.15.98:0/941072153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec06e750 con 0x7f54ec0b29f0 2024-09-10T20:59:09.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54e259c640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54ec15d5e0 0x7f54ec166a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54f0807640 1 -- 172.21.15.98:0/941072153 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f54ec06ea50 con 0x7f54ec15d5e0 2024-09-10T20:59:09.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54eb59e640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54ec14a170 0x7f54ec168ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54ead9d640 1 -- 172.21.15.98:0/941072153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54ec15d5e0 msgr2=0x7f54ec166a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54ead9d640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54ec15d5e0 0x7f54ec166a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54ead9d640 1 -- 172.21.15.98:0/941072153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54ec14a170 msgr2=0x7f54ec168ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54ead9d640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54ec14a170 0x7f54ec168ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f54ead9d640 1 -- 172.21.15.98:0/941072153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ec167400 con 0x7f54ec0b29f0 2024-09-10T20:59:09.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f54ead9d640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54ec0b29f0 0x7f54ec1474c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54d402a9f0 tx=0x7f54d4032f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f54eb59e640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54ec14a170 0x7f54ec168ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:09.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f54e3fff640 1 -- 172.21.15.98:0/941072153 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d4040070 con 0x7f54ec0b29f0 2024-09-10T20:59:09.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f54e3fff640 1 -- 172.21.15.98:0/941072153 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54d4003af0 con 0x7f54ec0b29f0 2024-09-10T20:59:09.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f54f0807640 1 -- 172.21.15.98:0/941072153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54ec147ec0 con 0x7f54ec0b29f0 2024-09-10T20:59:09.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f7e6b584640 1 Processor -- start 2024-09-10T20:59:09.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f7e6b584640 1 -- start start 2024-09-10T20:59:09.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.367+0000 7f0e6965b640 1 -- 172.21.15.98:0/2114102466 wait complete. 2024-09-10T20:59:09.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f0e6965b640 1 Processor -- start 2024-09-10T20:59:09.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f0e6965b640 1 -- start start 2024-09-10T20:59:09.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f6754352640 1 -- start start 2024-09-10T20:59:09.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f3da63a3640 1 Processor -- start 2024-09-10T20:59:09.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f3da63a3640 1 -- start start 2024-09-10T20:59:09.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f00c1423640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00b4037ee0 0x7f00b40d3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f00c1423640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00b403b910 0x7f00b4050940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f00c1423640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00b40e2770 0x7f00b4178300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f00c1423640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f00b4006f90 con 0x7f00b403b910 2024-09-10T20:59:09.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f00c1423640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f00b4006e10 con 0x7f00b4037ee0 2024-09-10T20:59:09.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f00c0c22640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00b40e2770 0x7f00b4178300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f00c0c22640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00b40e2770 0x7f00b4178300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:36090/0 (socket says 172.21.15.98:36090) 2024-09-10T20:59:09.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.369+0000 7f00c0c22640 1 -- 172.21.15.98:0/4115357741 learned_addr learned my addr 172.21.15.98:0/4115357741 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f00c1423640 1 -- 172.21.15.98:0/4115357741 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f00b4007110 con 0x7f00b40e2770 2024-09-10T20:59:09.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f00bb7fe640 1 --2- 172.21.15.98:0/4115357741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00b403b910 0x7f00b4050940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f54f0807640 1 -- 172.21.15.98:0/941072153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54ec1676a0 con 0x7f54ec0b29f0 2024-09-10T20:59:09.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f00bb7fe640 1 -- 172.21.15.98:0/4115357741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00b40e2770 msgr2=0x7f00b4178300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f00bb7fe640 1 --2- 172.21.15.98:0/4115357741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00b40e2770 0x7f00b4178300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f00bb7fe640 1 -- 172.21.15.98:0/4115357741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00b4037ee0 msgr2=0x7f00b40d3850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T20:59:09.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f00bb7fe640 1 --2- 172.21.15.98:0/4115357741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00b4037ee0 0x7f00b40d3850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f00bb7fe640 1 -- 172.21.15.98:0/4115357741 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b40512b0 con 0x7f00b403b910 2024-09-10T20:59:09.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f0e6965b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e50003aa0 0x7f0e6413fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f54e3fff640 1 -- 172.21.15.98:0/941072153 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d4045460 con 0x7f54ec0b29f0 2024-09-10T20:59:09.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f0e6965b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e641502a0 0x7f0e64142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f6754352640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6738004800 0x7f674400b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f6754352640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f674404c680 0x7f6744007f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f6754352640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6744008460 0x7f67440331c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f6754352640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6744036d90 con 0x7f674404c680 2024-09-10T20:59:09.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f6754352640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6744036c10 con 0x7f6744008460 2024-09-10T20:59:09.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.370+0000 7f6754352640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6744037550 con 0x7f6738004800 2024-09-10T20:59:09.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f67520ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6738004800 0x7f674400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f67528ef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6744008460 0x7f67440331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f0e63fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e50003aa0 0x7f0e6413fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f0e63fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e50003aa0 0x7f0e6413fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:35280/0 (socket says 172.21.15.98:35280) 2024-09-10T20:59:09.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f67528ef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6744008460 0x7f67440331c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:35266/0 (socket says 172.21.15.98:35266) 2024-09-10T20:59:09.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f67520ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6738004800 0x7f674400b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:36092/0 (socket says 172.21.15.98:36092) 2024-09-10T20:59:09.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f67520ee640 1 -- 172.21.15.98:0/659726366 learned_addr learned my addr 172.21.15.98:0/659726366 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f0e63fff640 1 -- 172.21.15.98:0/3143988290 learned_addr learned my addr 172.21.15.98:0/3143988290 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f0e6965b640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e64142bd0 0x7f0e641430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f67518ed640 1 --2- 172.21.15.98:0/659726366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f674404c680 0x7f6744007f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f3da63a3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8c003a20 0x7f3da014c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f0e6965b640 1 -- 172.21.15.98:0/3143988290 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0e640746a0 con 0x7f0e64142bd0 2024-09-10T20:59:09.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f7e6b584640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e640742e0 0x7f7e641471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f0e6965b640 1 -- 172.21.15.98:0/3143988290 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64074520 con 0x7f0e50003aa0 2024-09-10T20:59:09.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f0e6965b640 1 -- 172.21.15.98:0/3143988290 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0e640b29f0 con 0x7f0e641502a0 2024-09-10T20:59:09.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f0e637fe640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e641502a0 0x7f0e64142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f0e68e5a640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e64142bd0 0x7f0e641430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f7e6b584640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e640b2c00 0x7f7e64143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f7e6b584640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e64159b80 0x7f7e6415b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f3d9ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8c003a20 0x7f3da014c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f3614f13640 1 -- 172.21.15.98:0/3437500420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36080db200 con 0x7f3608008d90 2024-09-10T20:59:09.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f7e69320640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e640742e0 0x7f7e641471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f7e69320640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e640742e0 0x7f7e641471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:36098/0 (socket says 172.21.15.98:36098) 2024-09-10T20:59:09.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f7e69320640 1 -- 172.21.15.98:0/250796416 learned_addr learned my addr 172.21.15.98:0/250796416 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f67528ef640 1 -- 172.21.15.98:0/659726366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6738004800 msgr2=0x7f674400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f67528ef640 1 --2- 172.21.15.98:0/659726366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6738004800 0x7f674400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f67528ef640 1 -- 172.21.15.98:0/659726366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f674404c680 msgr2=0x7f6744007f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f7e69b21640 1 --2- 172.21.15.98:0/250796416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e64159b80 0x7f7e6415b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f67528ef640 1 --2- 172.21.15.98:0/659726366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f674404c680 0x7f6744007f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.371+0000 7f67528ef640 1 -- 172.21.15.98:0/659726366 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674400bea0 con 0x7f6744008460 2024-09-10T20:59:09.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f7e6b584640 1 -- 172.21.15.98:0/250796416 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7e640b3400 con 0x7f7e64159b80 2024-09-10T20:59:09.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f7e6b584640 1 -- 172.21.15.98:0/250796416 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7e640b3280 con 0x7f7e640b2c00 2024-09-10T20:59:09.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f7e6b584640 1 -- 172.21.15.98:0/250796416 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7e640b3580 con 0x7f7e640742e0 2024-09-10T20:59:09.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f3d9ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8c003a20 0x7f3da014c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58740/0 (socket says 172.21.15.98:58740) 2024-09-10T20:59:09.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f3da63a3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0155390 0x7f3da00c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f3d9ffff640 1 -- 172.21.15.98:0/1306201409 learned_addr learned my addr 172.21.15.98:0/1306201409 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f3da63a3640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da015c9b0 0x7f3da015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f3da63a3640 1 -- 172.21.15.98:0/1306201409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3da00cd020 con 0x7f3d8c003a20 2024-09-10T20:59:09.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f00bb7fe640 1 --2- 172.21.15.98:0/4115357741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00b403b910 0x7f00b4050940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f00a8031000 tx=0x7f00a8004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f7e68b1f640 1 --2- 172.21.15.98:0/250796416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e640b2c00 0x7f7e64143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f7e69320640 1 -- 172.21.15.98:0/250796416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e640b2c00 msgr2=0x7f7e64143e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f3da4940640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da015c9b0 0x7f3da015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f7e69320640 1 --2- 172.21.15.98:0/250796416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e640b2c00 0x7f7e64143e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f7e69320640 1 -- 172.21.15.98:0/250796416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e64159b80 msgr2=0x7f7e6415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f7e69320640 1 --2- 172.21.15.98:0/250796416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e64159b80 0x7f7e6415b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f7e69320640 1 -- 172.21.15.98:0/250796416 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e64147700 con 0x7f7e640742e0 2024-09-10T20:59:09.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f0e637fe640 1 -- 172.21.15.98:0/3143988290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e50003aa0 msgr2=0x7f0e6413fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f0e637fe640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e50003aa0 0x7f0e6413fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f0e637fe640 1 -- 172.21.15.98:0/3143988290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e64142bd0 msgr2=0x7f0e641430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f0e637fe640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e64142bd0 0x7f0e641430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f0e637fe640 1 -- 172.21.15.98:0/3143988290 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e64150a10 con 0x7f0e641502a0 2024-09-10T20:59:09.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f0e68e5a640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e64142bd0 0x7f0e641430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:09.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f0e637fe640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e641502a0 0x7f0e64142690 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0e6406c0e0 tx=0x7f0e4c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f3da63a3640 1 -- 172.21.15.98:0/1306201409 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3da00ccea0 con 0x7f3da015c9b0 2024-09-10T20:59:09.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f3da63a3640 1 -- 172.21.15.98:0/1306201409 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3da00c93b0 con 0x7f3da0155390 2024-09-10T20:59:09.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f7e69320640 1 --2- 172.21.15.98:0/250796416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e640742e0 0x7f7e641471c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e6002aae0 tx=0x7f7e60003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f0e60ff9640 1 -- 172.21.15.98:0/3143988290 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0052e0 con 0x7f0e641502a0 2024-09-10T20:59:09.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f0e60ff9640 1 -- 172.21.15.98:0/3143988290 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e4c01c990 con 0x7f0e641502a0 2024-09-10T20:59:09.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f67528ef640 1 --2- 172.21.15.98:0/659726366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6744008460 0x7f67440331c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f674400d4a0 tx=0x7f674800c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f0e6965b640 1 -- 172.21.15.98:0/3143988290 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e64150730 con 0x7f0e641502a0 2024-09-10T20:59:09.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f00b8ff9640 1 -- 172.21.15.98:0/4115357741 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8035020 con 0x7f00b403b910 2024-09-10T20:59:09.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f54e3fff640 1 -- 172.21.15.98:0/941072153 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f54d404d030 con 0x7f54ec0b29f0 2024-09-10T20:59:09.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f0e6965b640 1 -- 172.21.15.98:0/3143988290 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e64141e90 con 0x7f0e641502a0 2024-09-10T20:59:09.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f3d9ffff640 1 -- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0155390 msgr2=0x7f3da00c8e70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T20:59:09.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f3d9ffff640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0155390 0x7f3da00c8e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f3d9ffff640 1 -- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da015c9b0 msgr2=0x7f3da015ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f3d9ffff640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da015c9b0 0x7f3da015ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f3d9ffff640 1 -- 172.21.15.98:0/1306201409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da00ca7a0 con 0x7f3d8c003a20 2024-09-10T20:59:09.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.374+0000 7f007d394640 1 -- 172.21.15.98:0/1947216755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00780b4620 msgr2=0x7f00780741b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.374+0000 7f007d394640 1 --2- 172.21.15.98:0/1947216755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00780b4620 0x7f00780741b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f00780b4a90 tx=0x7f007002b490 comp rx=0 tx=0).stop 2024-09-10T20:59:09.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f35fbfff640 1 -- 172.21.15.98:0/3437500420 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35fc00a950 con 0x7f3608008d90 2024-09-10T20:59:09.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f35fbfff640 1 -- 172.21.15.98:0/3437500420 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc00ada0 con 0x7f3608008d90 2024-09-10T20:59:09.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.372+0000 7f35fbfff640 1 -- 172.21.15.98:0/3437500420 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f35fc012420 con 0x7f3608008d90 2024-09-10T20:59:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f35fbfff640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f35f4078070 0x7f35f407a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f360effd640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f35f4078070 0x7f35f407a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f35fbfff640 1 -- 172.21.15.98:0/3437500420 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f35fc09e060 con 0x7f3608008d90 2024-09-10T20:59:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.374+0000 7f35f9ffb640 1 -- 172.21.15.98:0/3437500420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3608050ce0 con 0x7f3608008d90 2024-09-10T20:59:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f7e59ffb640 1 -- 172.21.15.98:0/250796416 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60005220 con 0x7f7e640742e0 2024-09-10T20:59:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f7e59ffb640 1 -- 172.21.15.98:0/250796416 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e600054b0 con 0x7f7e640742e0 2024-09-10T20:59:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.374+0000 7f7e59ffb640 1 -- 172.21.15.98:0/250796416 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60004a00 con 0x7f7e640742e0 2024-09-10T20:59:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f0e60ff9640 1 -- 172.21.15.98:0/3143988290 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c01cde0 con 0x7f0e641502a0 2024-09-10T20:59:09.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f6742ffd640 1 -- 172.21.15.98:0/659726366 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748017070 con 0x7f6744008460 2024-09-10T20:59:09.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f6742ffd640 1 -- 172.21.15.98:0/659726366 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67480052d0 con 0x7f6744008460 2024-09-10T20:59:09.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f00c1423640 1 -- 172.21.15.98:0/4115357741 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b4051590 con 0x7f00b403b910 2024-09-10T20:59:09.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f00c1423640 1 -- 172.21.15.98:0/4115357741 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b4051b50 con 0x7f00b403b910 2024-09-10T20:59:09.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.373+0000 7f00b8ff9640 1 -- 172.21.15.98:0/4115357741 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00a8043070 con 0x7f00b403b910 2024-09-10T20:59:09.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.374+0000 7f00b8ff9640 1 -- 172.21.15.98:0/4115357741 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8004e80 con 0x7f00b403b910 2024-09-10T20:59:09.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.375+0000 7f0e6965b640 1 -- 172.21.15.98:0/3143988290 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e48005290 con 0x7f0e641502a0 2024-09-10T20:59:09.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.374+0000 7f54f0807640 1 -- 172.21.15.98:0/941072153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d0005290 con 0x7f54ec0b29f0 2024-09-10T20:59:09.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.375+0000 7f6742ffd640 1 -- 172.21.15.98:0/659726366 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748003850 con 0x7f6744008460 2024-09-10T20:59:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.374+0000 7f3da4940640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da015c9b0 0x7f3da015ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.375+0000 7f3d9ffff640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8c003a20 0x7f3da014c520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009c40 tx=0x7f3d88033e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.374+0000 7f7e6b584640 1 -- 172.21.15.98:0/250796416 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e64156940 con 0x7f7e640742e0 2024-09-10T20:59:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.375+0000 7f7e6b584640 1 -- 172.21.15.98:0/250796416 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e64144750 con 0x7f7e640742e0 2024-09-10T20:59:09.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.375+0000 7f6754352640 1 -- 172.21.15.98:0/659726366 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f674404d180 con 0x7f6744008460 2024-09-10T20:59:09.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.375+0000 7f6754352640 1 -- 172.21.15.98:0/659726366 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6744006620 con 0x7f6744008460 2024-09-10T20:59:09.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.375+0000 7f360effd640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f35f4078070 0x7f35f407a530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f36100661a0 tx=0x7f36100645a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.375+0000 7f007d394640 1 -- 172.21.15.98:0/1947216755 shutdown_connections 2024-09-10T20:59:09.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.375+0000 7f007d394640 1 --2- 172.21.15.98:0/1947216755 >> v2:172.21.15.190:3300/0 conn(0x7f00780b3750 0x7f00780cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.375+0000 7f007d394640 1 --2- 172.21.15.98:0/1947216755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00780b4620 0x7f00780741b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.375+0000 7f007d394640 1 -- 172.21.15.98:0/1947216755 >> 172.21.15.98:0/1947216755 conn(0x7f00780cfaf0 msgr2=0x7f00780cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.376+0000 7f54e3fff640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f54cc078040 0x7f54cc07a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.376+0000 7f54e3fff640 1 -- 172.21.15.98:0/941072153 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f54d40bb4b0 con 0x7f54ec0b29f0 2024-09-10T20:59:09.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.376+0000 7f7e6b584640 1 -- 172.21.15.98:0/250796416 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e6415e140 con 0x7f7e640742e0 2024-09-10T20:59:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.376+0000 7f00b8ff9640 1 -- 172.21.15.98:0/4115357741 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f00a803a040 con 0x7f00b403b910 2024-09-10T20:59:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.376+0000 7f00c1423640 1 -- 172.21.15.98:0/4115357741 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f009c005290 con 0x7f00b403b910 2024-09-10T20:59:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.376+0000 7f54e259c640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f54cc078040 0x7f54cc07a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.376+0000 7f007d394640 1 -- 172.21.15.98:0/1947216755 shutdown_connections 2024-09-10T20:59:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.376+0000 7f6754352640 1 -- 172.21.15.98:0/659726366 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f674404cf70 con 0x7f6744008460 2024-09-10T20:59:09.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.376+0000 7f0e60ff9640 1 -- 172.21.15.98:0/3143988290 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0e4c041a60 con 0x7f0e641502a0 2024-09-10T20:59:09.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.376+0000 7f007d394640 1 -- 172.21.15.98:0/1947216755 wait complete. 2024-09-10T20:59:09.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.377+0000 7f54e259c640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f54cc078040 0x7f54cc07a500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0cf480 tx=0x7f54dc007f10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.377+0000 7f00b8ff9640 1 --2- 172.21.15.98:0/4115357741 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f00a0078010 0x7f00a007a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.377+0000 7f7e59ffb640 1 -- 172.21.15.98:0/250796416 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7e60004ba0 con 0x7f7e640742e0 2024-09-10T20:59:09.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.377+0000 7f0e60ff9640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0e44078040 0x7f0e4407a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.377+0000 7f0e63fff640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0e44078040 0x7f0e4407a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.377+0000 7f0e60ff9640 1 -- 172.21.15.98:0/3143988290 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0e4c0a1bd0 con 0x7f0e641502a0 2024-09-10T20:59:09.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.377+0000 7f00b8ff9640 1 -- 172.21.15.98:0/4115357741 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f00a80babe0 con 0x7f00b403b910 2024-09-10T20:59:09.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.377+0000 7f00bbfff640 1 --2- 172.21.15.98:0/4115357741 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f00a0078010 0x7f00a007a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.378+0000 7f0e63fff640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0e44078040 0x7f0e4407a500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c530 tx=0x7f0e640b4030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.378+0000 7f00bbfff640 1 --2- 172.21.15.98:0/4115357741 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f00a0078010 0x7f00a007a4d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f00b00099c0 tx=0x7f00b0005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.378+0000 7f7e59ffb640 1 --2- 172.21.15.98:0/250796416 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7e48078040 0x7f7e4807a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.378+0000 7f7e59ffb640 1 -- 172.21.15.98:0/250796416 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7e600baf30 con 0x7f7e640742e0 2024-09-10T20:59:09.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.378+0000 7f7e68b1f640 1 --2- 172.21.15.98:0/250796416 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7e48078040 0x7f7e4807a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.378+0000 7f007d394640 1 Processor -- start 2024-09-10T20:59:09.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.378+0000 7f007d394640 1 -- start start 2024-09-10T20:59:09.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.378+0000 7f7e68b1f640 1 --2- 172.21.15.98:0/250796416 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7e48078040 0x7f7e4807a500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7e6415a410 tx=0x7f7e54009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f007d394640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0078073dd0 0x7f0078164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f007d394640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00780b3750 0x7f00780aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f007d394640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00780b4620 0x7f00780ab200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f007d394640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f00780cdf30 con 0x7f00780b3750 2024-09-10T20:59:09.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f007d394640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0078072770 con 0x7f0078073dd0 2024-09-10T20:59:09.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f007d394640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f00780ce0b0 con 0x7f00780b4620 2024-09-10T20:59:09.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f00767fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00780b3750 0x7f00780aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f00767fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00780b3750 0x7f00780aacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58766/0 (socket says 172.21.15.98:58766) 2024-09-10T20:59:09.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f00767fc640 1 -- 172.21.15.98:0/73296708 learned_addr learned my addr 172.21.15.98:0/73296708 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:09.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f0076ffd640 1 --2- 172.21.15.98:0/73296708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0078073dd0 0x7f0078164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f00777fe640 1 --2- 172.21.15.98:0/73296708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00780b4620 0x7f00780ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f00767fc640 1 -- 172.21.15.98:0/73296708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00780b4620 msgr2=0x7f00780ab200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f00767fc640 1 --2- 172.21.15.98:0/73296708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00780b4620 0x7f00780ab200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f00767fc640 1 -- 172.21.15.98:0/73296708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0078073dd0 msgr2=0x7f0078164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f00767fc640 1 --2- 172.21.15.98:0/73296708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0078073dd0 0x7f0078164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.379+0000 7f00767fc640 1 -- 172.21.15.98:0/73296708 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0078201300 con 0x7f00780b3750 2024-09-10T20:59:09.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.380+0000 7f00767fc640 1 --2- 172.21.15.98:0/73296708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00780b3750 0x7f00780aacc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f007000a040 tx=0x7f0070003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.380+0000 7f0e60ff9640 1 -- 172.21.15.98:0/3143988290 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0e4c0a1e20 con 0x7f0e641502a0 2024-09-10T20:59:09.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.381+0000 7f54e3fff640 1 -- 172.21.15.98:0/941072153 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f54d4083c60 con 0x7f54ec0b29f0 2024-09-10T20:59:09.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.381+0000 7f3d9cff9640 1 -- 172.21.15.98:0/1306201409 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88038050 con 0x7f3d8c003a20 2024-09-10T20:59:09.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.381+0000 7f35fbfff640 1 -- 172.21.15.98:0/3437500420 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f35fc061fa0 con 0x7f3608008d90 2024-09-10T20:59:09.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.381+0000 7f3da63a3640 1 -- 172.21.15.98:0/1306201409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da00caa80 con 0x7f3d8c003a20 2024-09-10T20:59:09.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.382+0000 7f3da63a3640 1 -- 172.21.15.98:0/1306201409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da00c8950 con 0x7f3d8c003a20 2024-09-10T20:59:09.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.382+0000 7f005f7fe640 1 -- 172.21.15.98:0/73296708 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070041030 con 0x7f00780b3750 2024-09-10T20:59:09.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.382+0000 7f007d394640 1 -- 172.21.15.98:0/73296708 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00782015c0 con 0x7f00780b3750 2024-09-10T20:59:09.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.382+0000 7f007d394640 1 -- 172.21.15.98:0/73296708 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0078201b80 con 0x7f00780b3750 2024-09-10T20:59:09.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.382+0000 7f6742ffd640 1 -- 172.21.15.98:0/659726366 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f67480039f0 con 0x7f6744008460 2024-09-10T20:59:09.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.383+0000 7f007d394640 1 -- 172.21.15.98:0/73296708 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00780cd9f0 con 0x7f00780b3750 2024-09-10T20:59:09.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.383+0000 7f3da63a3640 1 -- 172.21.15.98:0/1306201409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da00b0c20 con 0x7f3d8c003a20 2024-09-10T20:59:09.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.383+0000 7f6742ffd640 1 --2- 172.21.15.98:0/659726366 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6730078040 0x7f673007a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.383+0000 7f67520ee640 1 --2- 172.21.15.98:0/659726366 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6730078040 0x7f673007a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.383+0000 7f6742ffd640 1 -- 172.21.15.98:0/659726366 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6748099450 con 0x7f6744008460 2024-09-10T20:59:09.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.384+0000 7f67520ee640 1 --2- 172.21.15.98:0/659726366 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6730078040 0x7f673007a500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f674c07adb0 tx=0x7f674c070040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.386+0000 7f6742ffd640 1 -- 172.21.15.98:0/659726366 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6748021080 con 0x7f6744008460 2024-09-10T20:59:09.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.386+0000 7f7e59ffb640 1 -- 172.21.15.98:0/250796416 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7e6003f030 con 0x7f7e640742e0 2024-09-10T20:59:09.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.387+0000 7f005f7fe640 1 -- 172.21.15.98:0/73296708 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f007000a0d0 con 0x7f00780b3750 2024-09-10T20:59:09.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.387+0000 7f005f7fe640 1 -- 172.21.15.98:0/73296708 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007000a490 con 0x7f00780b3750 2024-09-10T20:59:09.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.387+0000 7f00b8ff9640 1 -- 172.21.15.98:0/4115357741 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f00a80832f0 con 0x7f00b403b910 2024-09-10T20:59:09.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.388+0000 7f3d9cff9640 1 -- 172.21.15.98:0/1306201409 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d88039040 con 0x7f3d8c003a20 2024-09-10T20:59:09.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.388+0000 7f3d9cff9640 1 -- 172.21.15.98:0/1306201409 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88004020 con 0x7f3d8c003a20 2024-09-10T20:59:09.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.391+0000 7f005f7fe640 1 -- 172.21.15.98:0/73296708 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f007000a630 con 0x7f00780b3750 2024-09-10T20:59:09.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.393+0000 7f3d9cff9640 1 -- 172.21.15.98:0/1306201409 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3d880041c0 con 0x7f3d8c003a20 2024-09-10T20:59:09.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.391+0000 7f005f7fe640 1 --2- 172.21.15.98:0/73296708 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0064077f70 0x7f006407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.391+0000 7f005f7fe640 1 -- 172.21.15.98:0/73296708 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f00700bb470 con 0x7f00780b3750 2024-09-10T20:59:09.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.393+0000 7f0076ffd640 1 --2- 172.21.15.98:0/73296708 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0064077f70 0x7f006407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.393+0000 7f3d9cff9640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3d80077f70 0x7f3d8007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:09.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.394+0000 7f3d9cff9640 1 -- 172.21.15.98:0/1306201409 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3d880bba90 con 0x7f3d8c003a20 2024-09-10T20:59:09.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.394+0000 7f3d9f7fe640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3d80077f70 0x7f3d8007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:09.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.394+0000 7f0076ffd640 1 --2- 172.21.15.98:0/73296708 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0064077f70 0x7f006407a430 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0068001b70 tx=0x7f0068001b00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.395+0000 7f3d9f7fe640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3d80077f70 0x7f3d8007a430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3d90001bc0 tx=0x7f3d90001830 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:09.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.396+0000 7f3d9cff9640 1 -- 172.21.15.98:0/1306201409 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3d880842c0 con 0x7f3d8c003a20 2024-09-10T20:59:09.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.405+0000 7f005f7fe640 1 -- 172.21.15.98:0/73296708 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f007004b070 con 0x7f00780b3750 2024-09-10T20:59:09.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.801+0000 7f35f9ffb640 1 -- 172.21.15.98:0/3437500420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f36080343c0 con 0x7f3608008d90 2024-09-10T20:59:09.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.802+0000 7f35fbfff640 1 -- 172.21.15.98:0/3437500420 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f35fc0618a0 con 0x7f3608008d90 2024-09-10T20:59:09.806 INFO:teuthology.orchestra.run.smithi098.stdout:64424509449 2024-09-10T20:59:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.805+0000 7f3614f13640 1 -- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f35f4078070 msgr2=0x7f35f407a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.806+0000 7f3614f13640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f35f4078070 0x7f35f407a530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f36100661a0 tx=0x7f36100645a0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.806+0000 7f3614f13640 1 -- 172.21.15.98:0/3437500420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3608008d90 msgr2=0x7f360817ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.806+0000 7f3614f13640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3608008d90 0x7f360817ca60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00c780 tx=0x7f35fc00cc40 comp rx=0 tx=0).stop 2024-09-10T20:59:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.806+0000 7f3614f13640 1 -- 172.21.15.98:0/3437500420 shutdown_connections 2024-09-10T20:59:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.806+0000 7f3614f13640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f35f4078070 0x7f35f407a530 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.806+0000 7f3614f13640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3608034ba0 0x7f3608051f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.806+0000 7f3614f13640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3608034650 0x7f36080dd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.806+0000 7f3614f13640 1 --2- 172.21.15.98:0/3437500420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3608008d90 0x7f360817ca60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.807+0000 7f3614f13640 1 -- 172.21.15.98:0/3437500420 >> 172.21.15.98:0/3437500420 conn(0x7f360801e890 msgr2=0x7f36080e1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.807+0000 7f3614f13640 1 -- 172.21.15.98:0/3437500420 shutdown_connections 2024-09-10T20:59:09.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.807+0000 7f3614f13640 1 -- 172.21.15.98:0/3437500420 wait complete. 2024-09-10T20:59:09.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.807+0000 7f7e6b584640 1 -- 172.21.15.98:0/250796416 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7e64068c20 con 0x7f7e640742e0 2024-09-10T20:59:09.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.807+0000 7f6754352640 1 -- 172.21.15.98:0/659726366 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6744055f90 con 0x7f6744008460 2024-09-10T20:59:09.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.808+0000 7f7e59ffb640 1 -- 172.21.15.98:0/250796416 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7e60083760 con 0x7f7e640742e0 2024-09-10T20:59:09.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.808+0000 7f6742ffd640 1 -- 172.21.15.98:0/659726366 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f674806a3a0 con 0x7f6744008460 2024-09-10T20:59:09.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.808+0000 7f54f0807640 1 -- 172.21.15.98:0/941072153 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f54d0005580 con 0x7f54ec0b29f0 2024-09-10T20:59:09.814 INFO:teuthology.orchestra.run.smithi098.stdout:60129542152 2024-09-10T20:59:09.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.808+0000 7f54e3fff640 1 -- 172.21.15.98:0/941072153 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f54d4083560 con 0x7f54ec0b29f0 2024-09-10T20:59:09.814 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2024-09-10T20:59:09.815 INFO:teuthology.orchestra.run.smithi098.stdout:64424509449 2024-09-10T20:59:09.816 INFO:teuthology.orchestra.run.smithi098.stdout:60129542152 2024-09-10T20:59:09.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.808+0000 7f0e6965b640 1 -- 172.21.15.98:0/3143988290 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0e48005850 con 0x7f0e641502a0 2024-09-10T20:59:09.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.809+0000 7f0e60ff9640 1 -- 172.21.15.98:0/3143988290 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0e4c021030 con 0x7f0e641502a0 2024-09-10T20:59:09.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.811+0000 7f00c1423640 1 -- 172.21.15.98:0/4115357741 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f009c005580 con 0x7f00b403b910 2024-09-10T20:59:09.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f6754352640 1 -- 172.21.15.98:0/659726366 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6730078040 msgr2=0x7f673007a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f6754352640 1 --2- 172.21.15.98:0/659726366 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6730078040 0x7f673007a500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f674c07adb0 tx=0x7f674c070040 comp rx=0 tx=0).stop 2024-09-10T20:59:09.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f6754352640 1 -- 172.21.15.98:0/659726366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6744008460 msgr2=0x7f67440331c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f6754352640 1 --2- 172.21.15.98:0/659726366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6744008460 0x7f67440331c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f674400d4a0 tx=0x7f674800c820 comp rx=0 tx=0).stop 2024-09-10T20:59:09.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f7e6b584640 1 -- 172.21.15.98:0/250796416 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7e48078040 msgr2=0x7f7e4807a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f7e6b584640 1 --2- 172.21.15.98:0/250796416 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7e48078040 0x7f7e4807a500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7e6415a410 tx=0x7f7e54009040 comp rx=0 tx=0).stop 2024-09-10T20:59:09.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f7e6b584640 1 -- 172.21.15.98:0/250796416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e640742e0 msgr2=0x7f7e641471c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f7e6b584640 1 --2- 172.21.15.98:0/250796416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e640742e0 0x7f7e641471c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e6002aae0 tx=0x7f7e60003b90 comp rx=0 tx=0).stop 2024-09-10T20:59:09.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f7e6b584640 1 -- 172.21.15.98:0/250796416 shutdown_connections 2024-09-10T20:59:09.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f7e6b584640 1 --2- 172.21.15.98:0/250796416 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7e48078040 0x7f7e4807a500 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f7e6b584640 1 --2- 172.21.15.98:0/250796416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e64159b80 0x7f7e6415b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f7e6b584640 1 --2- 172.21.15.98:0/250796416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e640b2c00 0x7f7e64143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f7e6b584640 1 --2- 172.21.15.98:0/250796416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e640742e0 0x7f7e641471c0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e6002aae0 tx=0x7f7e60003b90 comp rx=0 tx=0).stop 2024-09-10T20:59:09.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f7e6b584640 1 -- 172.21.15.98:0/250796416 >> 172.21.15.98:0/250796416 conn(0x7f7e6406ca50 msgr2=0x7f7e640acd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f6754352640 1 -- 172.21.15.98:0/659726366 shutdown_connections 2024-09-10T20:59:09.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f6754352640 1 --2- 172.21.15.98:0/659726366 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6730078040 0x7f673007a500 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f6754352640 1 --2- 172.21.15.98:0/659726366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6744008460 0x7f67440331c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f6754352640 1 --2- 172.21.15.98:0/659726366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f674404c680 0x7f6744007f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f6754352640 1 --2- 172.21.15.98:0/659726366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6738004800 0x7f674400b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f6754352640 1 -- 172.21.15.98:0/659726366 >> 172.21.15.98:0/659726366 conn(0x7f674401e890 msgr2=0x7f674404fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f54f0807640 1 -- 172.21.15.98:0/941072153 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f54cc078040 msgr2=0x7f54cc07a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f00b8ff9640 1 -- 172.21.15.98:0/4115357741 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f00a8082bf0 con 0x7f00b403b910 2024-09-10T20:59:09.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f6754352640 1 -- 172.21.15.98:0/659726366 shutdown_connections 2024-09-10T20:59:09.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f6754352640 1 -- 172.21.15.98:0/659726366 wait complete. 2024-09-10T20:59:09.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f7e6b584640 1 -- 172.21.15.98:0/250796416 shutdown_connections 2024-09-10T20:59:09.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f54f0807640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f54cc078040 0x7f54cc07a500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0cf480 tx=0x7f54dc007f10 comp rx=0 tx=0).stop 2024-09-10T20:59:09.822 INFO:teuthology.orchestra.run.smithi098.stdout:60129542152 2024-09-10T20:59:09.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f54f0807640 1 -- 172.21.15.98:0/941072153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54ec0b29f0 msgr2=0x7f54ec1474c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f54f0807640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54ec0b29f0 0x7f54ec1474c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54d402a9f0 tx=0x7f54d4032f00 comp rx=0 tx=0).stop 2024-09-10T20:59:09.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f54f0807640 1 -- 172.21.15.98:0/941072153 shutdown_connections 2024-09-10T20:59:09.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f54f0807640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f54cc078040 0x7f54cc07a500 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f54f0807640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54ec14a170 0x7f54ec168ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f3da63a3640 1 -- 172.21.15.98:0/1306201409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3da00d1090 con 0x7f3d8c003a20 2024-09-10T20:59:09.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f7e6b584640 1 -- 172.21.15.98:0/250796416 wait complete. 2024-09-10T20:59:09.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f54f0807640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54ec15d5e0 0x7f54ec166a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f54f0807640 1 --2- 172.21.15.98:0/941072153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54ec0b29f0 0x7f54ec1474c0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54d402a9f0 tx=0x7f54d4032f00 comp rx=0 tx=0).stop 2024-09-10T20:59:09.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.812+0000 7f54f0807640 1 -- 172.21.15.98:0/941072153 >> 172.21.15.98:0/941072153 conn(0x7f54ec06ce40 msgr2=0x7f54ec15ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.813+0000 7f54f0807640 1 -- 172.21.15.98:0/941072153 shutdown_connections 2024-09-10T20:59:09.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.813+0000 7f54f0807640 1 -- 172.21.15.98:0/941072153 wait complete. 2024-09-10T20:59:09.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.813+0000 7f0e6965b640 1 -- 172.21.15.98:0/3143988290 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0e44078040 msgr2=0x7f0e4407a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.813+0000 7f0e6965b640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0e44078040 0x7f0e4407a500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c530 tx=0x7f0e640b4030 comp rx=0 tx=0).stop 2024-09-10T20:59:09.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.813+0000 7f0e6965b640 1 -- 172.21.15.98:0/3143988290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e641502a0 msgr2=0x7f0e64142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.813+0000 7f3d9cff9640 1 -- 172.21.15.98:0/1306201409 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d88083bc0 con 0x7f3d8c003a20 2024-09-10T20:59:09.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.813+0000 7f0e6965b640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e641502a0 0x7f0e64142690 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0e6406c0e0 tx=0x7f0e4c00c800 comp rx=0 tx=0).stop 2024-09-10T20:59:09.826 INFO:teuthology.orchestra.run.smithi098.stdout:64424509449 2024-09-10T20:59:09.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.813+0000 7f0e6965b640 1 -- 172.21.15.98:0/3143988290 shutdown_connections 2024-09-10T20:59:09.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.814+0000 7f0e6965b640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0e44078040 0x7f0e4407a500 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.814+0000 7f0e6965b640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e64142bd0 0x7f0e641430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.814+0000 7f0e6965b640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e641502a0 0x7f0e64142690 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.814+0000 7f0e6965b640 1 --2- 172.21.15.98:0/3143988290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e50003aa0 0x7f0e6413fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.814+0000 7f0e6965b640 1 -- 172.21.15.98:0/3143988290 >> 172.21.15.98:0/3143988290 conn(0x7f0e6406ce40 msgr2=0x7f0e6415cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.814+0000 7f0e6965b640 1 -- 172.21.15.98:0/3143988290 shutdown_connections 2024-09-10T20:59:09.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.814+0000 7f0e6965b640 1 -- 172.21.15.98:0/3143988290 wait complete. 2024-09-10T20:59:09.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.816+0000 7f00c1423640 1 -- 172.21.15.98:0/4115357741 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f00a0078010 msgr2=0x7f00a007a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.816+0000 7f00c1423640 1 --2- 172.21.15.98:0/4115357741 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f00a0078010 0x7f00a007a4d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f00b00099c0 tx=0x7f00b0005f90 comp rx=0 tx=0).stop 2024-09-10T20:59:09.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.816+0000 7f00c1423640 1 -- 172.21.15.98:0/4115357741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00b403b910 msgr2=0x7f00b4050940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.816+0000 7f00c1423640 1 --2- 172.21.15.98:0/4115357741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00b403b910 0x7f00b4050940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f00a8031000 tx=0x7f00a8004820 comp rx=0 tx=0).stop 2024-09-10T20:59:09.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.816+0000 7f00c1423640 1 -- 172.21.15.98:0/4115357741 shutdown_connections 2024-09-10T20:59:09.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.816+0000 7f00c1423640 1 --2- 172.21.15.98:0/4115357741 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f00a0078010 0x7f00a007a4d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.816+0000 7f00c1423640 1 --2- 172.21.15.98:0/4115357741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00b40e2770 0x7f00b4178300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.817+0000 7f00c1423640 1 --2- 172.21.15.98:0/4115357741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00b403b910 0x7f00b4050940 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.817+0000 7f00c1423640 1 --2- 172.21.15.98:0/4115357741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00b4037ee0 0x7f00b40d3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.817+0000 7f00c1423640 1 -- 172.21.15.98:0/4115357741 >> 172.21.15.98:0/4115357741 conn(0x7f00b403a3a0 msgr2=0x7f00b40e4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.817+0000 7f00c1423640 1 -- 172.21.15.98:0/4115357741 shutdown_connections 2024-09-10T20:59:09.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.817+0000 7f00c1423640 1 -- 172.21.15.98:0/4115357741 wait complete. 2024-09-10T20:59:09.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.817+0000 7f3da63a3640 1 -- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3d80077f70 msgr2=0x7f3d8007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.817+0000 7f3da63a3640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3d80077f70 0x7f3d8007a430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3d90001bc0 tx=0x7f3d90001830 comp rx=0 tx=0).stop 2024-09-10T20:59:09.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.818+0000 7f3da63a3640 1 -- 172.21.15.98:0/1306201409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8c003a20 msgr2=0x7f3da014c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.818+0000 7f3da63a3640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8c003a20 0x7f3da014c520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009c40 tx=0x7f3d88033e90 comp rx=0 tx=0).stop 2024-09-10T20:59:09.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.818+0000 7f3da63a3640 1 -- 172.21.15.98:0/1306201409 shutdown_connections 2024-09-10T20:59:09.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.818+0000 7f3da63a3640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3d80077f70 0x7f3d8007a430 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.818+0000 7f3da63a3640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da015c9b0 0x7f3da015ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.818+0000 7f3da63a3640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da0155390 0x7f3da00c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.818+0000 7f3da63a3640 1 --2- 172.21.15.98:0/1306201409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8c003a20 0x7f3da014c520 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.818+0000 7f3da63a3640 1 -- 172.21.15.98:0/1306201409 >> 172.21.15.98:0/1306201409 conn(0x7f3da00d0940 msgr2=0x7f3da00b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.819+0000 7f3da63a3640 1 -- 172.21.15.98:0/1306201409 shutdown_connections 2024-09-10T20:59:09.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.819+0000 7f3da63a3640 1 -- 172.21.15.98:0/1306201409 wait complete. 2024-09-10T20:59:09.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.822+0000 7f007d394640 1 -- 172.21.15.98:0/73296708 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f00780aaaa0 con 0x7f00780b3750 2024-09-10T20:59:09.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.823+0000 7f005f7fe640 1 -- 172.21.15.98:0/73296708 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f00700835c0 con 0x7f00780b3750 2024-09-10T20:59:09.832 INFO:teuthology.orchestra.run.smithi098.stdout:60129542152 2024-09-10T20:59:09.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.827+0000 7f007d394640 1 -- 172.21.15.98:0/73296708 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0064077f70 msgr2=0x7f006407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.827+0000 7f007d394640 1 --2- 172.21.15.98:0/73296708 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0064077f70 0x7f006407a430 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0068001b70 tx=0x7f0068001b00 comp rx=0 tx=0).stop 2024-09-10T20:59:09.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.827+0000 7f007d394640 1 -- 172.21.15.98:0/73296708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00780b3750 msgr2=0x7f00780aacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:09.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.827+0000 7f007d394640 1 --2- 172.21.15.98:0/73296708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00780b3750 0x7f00780aacc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f007000a040 tx=0x7f0070003f10 comp rx=0 tx=0).stop 2024-09-10T20:59:09.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.828+0000 7f007d394640 1 -- 172.21.15.98:0/73296708 shutdown_connections 2024-09-10T20:59:09.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.828+0000 7f007d394640 1 --2- 172.21.15.98:0/73296708 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0064077f70 0x7f006407a430 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.828+0000 7f007d394640 1 --2- 172.21.15.98:0/73296708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00780b4620 0x7f00780ab200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.828+0000 7f007d394640 1 --2- 172.21.15.98:0/73296708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00780b3750 0x7f00780aacc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.828+0000 7f007d394640 1 --2- 172.21.15.98:0/73296708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0078073dd0 0x7f0078164770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:09.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.828+0000 7f007d394640 1 -- 172.21.15.98:0/73296708 >> 172.21.15.98:0/73296708 conn(0x7f00780cfaf0 msgr2=0x7f00780d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:09.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.828+0000 7f007d394640 1 -- 172.21.15.98:0/73296708 shutdown_connections 2024-09-10T20:59:09.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:09.828+0000 7f007d394640 1 -- 172.21.15.98:0/73296708 wait complete. 2024-09-10T20:59:09.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.4 2024-09-10T20:59:09.839 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.5 2024-09-10T20:59:09.840 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.0 2024-09-10T20:59:09.841 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.7 2024-09-10T20:59:09.842 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.3 2024-09-10T20:59:09.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.1 2024-09-10T20:59:09.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.6 2024-09-10T20:59:09.849 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.2 2024-09-10T20:59:10.837 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T20:59:10.840 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T20:59:10.842 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T20:59:10.842 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T20:59:10.843 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T20:59:10.845 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T20:59:10.845 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T20:59:10.850 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T20:59:11.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.002+0000 7fabc69c6640 1 -- 172.21.15.98:0/1076730177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc00cd9f0 msgr2=0x7fabc00cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.002+0000 7fabc69c6640 1 --2- 172.21.15.98:0/1076730177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc00cd9f0 0x7fabc00cde20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fabc00b27e0 tx=0x7fabb802aaf0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.002+0000 7fabc69c6640 1 -- 172.21.15.98:0/1076730177 shutdown_connections 2024-09-10T20:59:11.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.002+0000 7fabc69c6640 1 --2- 172.21.15.98:0/1076730177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc00cd9f0 0x7fabc00cde20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.002+0000 7fabc69c6640 1 --2- 172.21.15.98:0/1076730177 >> v2:172.21.15.98:3300/0 conn(0x7fabc015db10 0x7fabc015df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.002+0000 7fabc69c6640 1 -- 172.21.15.98:0/1076730177 >> 172.21.15.98:0/1076730177 conn(0x7fabc00cfaf0 msgr2=0x7fabc00cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.002+0000 7f5806fa6640 1 -- 172.21.15.98:0/1763793911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f80366f0 msgr2=0x7f57f801c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.002+0000 7f5806fa6640 1 --2- 172.21.15.98:0/1763793911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f80366f0 0x7f57f801c620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5800069f00 tx=0x7f58000a5bc0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.003+0000 7fabc69c6640 1 -- 172.21.15.98:0/1076730177 shutdown_connections 2024-09-10T20:59:11.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.003+0000 7fabc69c6640 1 -- 172.21.15.98:0/1076730177 wait complete. 2024-09-10T20:59:11.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.003+0000 7f5806fa6640 1 -- 172.21.15.98:0/1763793911 shutdown_connections 2024-09-10T20:59:11.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.003+0000 7f5806fa6640 1 --2- 172.21.15.98:0/1763793911 >> v2:172.21.15.98:3300/0 conn(0x7f57f8005a60 0x7f57f801cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.003+0000 7f5806fa6640 1 --2- 172.21.15.98:0/1763793911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f80366f0 0x7f57f801c620 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.003+0000 7f5806fa6640 1 -- 172.21.15.98:0/1763793911 >> 172.21.15.98:0/1763793911 conn(0x7f57f8054560 msgr2=0x7f57f8054940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.002+0000 7fd1ad59a640 1 -- 172.21.15.98:0/2255108554 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b00ab420 con 0x7fd1a801c890 2024-09-10T20:59:11.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.003+0000 7fd1af59e640 1 -- 172.21.15.98:0/2255108554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a801c890 msgr2=0x7fd1a801cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.003+0000 7fd1af59e640 1 --2- 172.21.15.98:0/2255108554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a801c890 0x7fd1a801cc70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd1b006b130 tx=0x7fd1b006b080 comp rx=0 tx=0).stop 2024-09-10T20:59:11.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.003+0000 7fd1af59e640 1 -- 172.21.15.98:0/2255108554 shutdown_connections 2024-09-10T20:59:11.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.003+0000 7fd1af59e640 1 --2- 172.21.15.98:0/2255108554 >> v2:172.21.15.98:3300/0 conn(0x7fd1a8005a60 0x7fd1a8035fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.003+0000 7fd1af59e640 1 --2- 172.21.15.98:0/2255108554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a801c890 0x7fd1a801cc70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.003+0000 7fd1af59e640 1 -- 172.21.15.98:0/2255108554 >> 172.21.15.98:0/2255108554 conn(0x7fd1a8054560 msgr2=0x7fd1a8054940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.003+0000 7f5806fa6640 1 -- 172.21.15.98:0/1763793911 shutdown_connections 2024-09-10T20:59:11.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.003+0000 7f5806fa6640 1 -- 172.21.15.98:0/1763793911 wait complete. 2024-09-10T20:59:11.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.004+0000 7fd1af59e640 1 -- 172.21.15.98:0/2255108554 shutdown_connections 2024-09-10T20:59:11.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.004+0000 7fd1af59e640 1 -- 172.21.15.98:0/2255108554 wait complete. 2024-09-10T20:59:11.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.005+0000 7f5806fa6640 1 Processor -- start 2024-09-10T20:59:11.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.005+0000 7f5806fa6640 1 -- start start 2024-09-10T20:59:11.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.005+0000 7fabc69c6640 1 Processor -- start 2024-09-10T20:59:11.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.005+0000 7fabc69c6640 1 -- start start 2024-09-10T20:59:11.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.006+0000 7fd1af59e640 1 Processor -- start 2024-09-10T20:59:11.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.006+0000 7fd1af59e640 1 -- start start 2024-09-10T20:59:11.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7f307227f640 1 -- 172.21.15.98:0/3741091072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3054003a50 msgr2=0x7f3054003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7f307227f640 1 --2- 172.21.15.98:0/3741091072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3054003a50 0x7f3054003e30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3060006f80 tx=0x7f3054005830 comp rx=0 tx=0).stop 2024-09-10T20:59:11.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7fabc69c6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc0070c80 0x7fabc013bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7f307081c640 1 -- 172.21.15.98:0/3741091072 reap_dead start 2024-09-10T20:59:11.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7fabc69c6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc00cd9f0 0x7fabc013f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.006+0000 7f5806fa6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f8005a60 0x7f57f80509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7f307227f640 1 -- 172.21.15.98:0/3741091072 shutdown_connections 2024-09-10T20:59:11.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7f307227f640 1 -- 172.21.15.98:0/3741091072 >> 172.21.15.98:0/3741091072 conn(0x7f306c0b2770 msgr2=0x7f306c0b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7fabc69c6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabc015db10 0x7fabc01f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7fabbffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc0070c80 0x7fabc013bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7fabbffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc0070c80 0x7fabc013bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:35412/0 (socket says 172.21.15.98:35412) 2024-09-10T20:59:11.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7fabbf7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc00cd9f0 0x7fabc013f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7fabbf7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc00cd9f0 0x7fabc013f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:32874/0 (socket says 172.21.15.98:32874) 2024-09-10T20:59:11.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7f307227f640 1 -- 172.21.15.98:0/3741091072 shutdown_connections 2024-09-10T20:59:11.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fabbf7fe640 1 -- 172.21.15.98:0/42870002 learned_addr learned my addr 172.21.15.98:0/42870002 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:11.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fabc69c6640 1 -- 172.21.15.98:0/42870002 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fabc00d0610 con 0x7fabc015db10 2024-09-10T20:59:11.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fabc69c6640 1 -- 172.21.15.98:0/42870002 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fabc00b42d0 con 0x7fabc00cd9f0 2024-09-10T20:59:11.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fabc69c6640 1 -- 172.21.15.98:0/42870002 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fabc00d0790 con 0x7fabc0070c80 2024-09-10T20:59:11.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fabc4f63640 1 --2- 172.21.15.98:0/42870002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabc015db10 0x7fabc01f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fd1af59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8005a60 0x7fd1a80e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.006+0000 7f5806fa6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f800ba00 0x7f57f80df2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.006+0000 7f5806fa6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57f80366f0 0x7f57f80e0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.006+0000 7f5806fa6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f57f8009e90 con 0x7f57f800ba00 2024-09-10T20:59:11.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.006+0000 7f5806fa6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f57f8009d10 con 0x7f57f8005a60 2024-09-10T20:59:11.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.006+0000 7f5806fa6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f57f800a010 con 0x7f57f80366f0 2024-09-10T20:59:11.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7f58067a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57f80366f0 0x7f57f80e0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7f58057a3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f800ba00 0x7f57f80df2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7f58067a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57f80366f0 0x7f57f80e0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:35406/0 (socket says 172.21.15.98:35406) 2024-09-10T20:59:11.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.007+0000 7f58067a5640 1 -- 172.21.15.98:0/2710856648 learned_addr learned my addr 172.21.15.98:0/2710856648 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:11.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7f5805fa4640 1 --2- 172.21.15.98:0/2710856648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f8005a60 0x7f57f80509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fd1af59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a801c890 0x7fd1a804caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fd1af59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a8036520 0x7fd1a804d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fd1ae59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8005a60 0x7fd1a80e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7f5805fa4640 1 -- 172.21.15.98:0/2710856648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57f80366f0 msgr2=0x7f57f80e0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7f5805fa4640 1 --2- 172.21.15.98:0/2710856648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57f80366f0 0x7f57f80e0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7f5805fa4640 1 -- 172.21.15.98:0/2710856648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f800ba00 msgr2=0x7f57f80df2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fd1ae59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8005a60 0x7fd1a80e1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:32892/0 (socket says 172.21.15.98:32892) 2024-09-10T20:59:11.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fd1ae59c640 1 -- 172.21.15.98:0/1688588162 learned_addr learned my addr 172.21.15.98:0/1688588162 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:11.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fabbffff640 1 -- 172.21.15.98:0/42870002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc00cd9f0 msgr2=0x7fabc013f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fabbffff640 1 --2- 172.21.15.98:0/42870002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc00cd9f0 0x7fabc013f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fabbffff640 1 -- 172.21.15.98:0/42870002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabc015db10 msgr2=0x7fabc01f3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fabbffff640 1 --2- 172.21.15.98:0/42870002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabc015db10 0x7fabc01f3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.008+0000 7fabbffff640 1 -- 172.21.15.98:0/42870002 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc013e430 con 0x7fabc0070c80 2024-09-10T20:59:11.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7fabbffff640 1 --2- 172.21.15.98:0/42870002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc0070c80 0x7fabc013bd20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fabb4007fb0 tx=0x7fabb400eae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7fd1add9b640 1 --2- 172.21.15.98:0/1688588162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a801c890 0x7fd1a804caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7fd1aed9d640 1 --2- 172.21.15.98:0/1688588162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a8036520 0x7fd1a804d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7fd1af59e640 1 -- 172.21.15.98:0/1688588162 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a800aa60 con 0x7fd1a801c890 2024-09-10T20:59:11.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7fd1af59e640 1 -- 172.21.15.98:0/1688588162 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a800bc10 con 0x7fd1a8005a60 2024-09-10T20:59:11.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7f5805fa4640 1 --2- 172.21.15.98:0/2710856648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f800ba00 0x7f57f80df2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7f5805fa4640 1 -- 172.21.15.98:0/2710856648 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f80d84d0 con 0x7f57f8005a60 2024-09-10T20:59:11.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7fd1af59e640 1 -- 172.21.15.98:0/1688588162 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd1a800abe0 con 0x7fd1a8036520 2024-09-10T20:59:11.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7f58057a3640 1 --2- 172.21.15.98:0/2710856648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f800ba00 0x7f57f80df2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:59:11.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.010+0000 7fd1add9b640 1 -- 172.21.15.98:0/1688588162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a8036520 msgr2=0x7fd1a804d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.010+0000 7fd1add9b640 1 --2- 172.21.15.98:0/1688588162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a8036520 0x7fd1a804d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.010+0000 7fd1add9b640 1 -- 172.21.15.98:0/1688588162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8005a60 msgr2=0x7fd1a80e1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.010+0000 7fd1add9b640 1 --2- 172.21.15.98:0/1688588162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8005a60 0x7fd1a80e1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.010+0000 7fd1add9b640 1 -- 172.21.15.98:0/1688588162 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a80dff90 con 0x7fd1a801c890 2024-09-10T20:59:11.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7fabbcff9640 1 -- 172.21.15.98:0/42870002 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4005270 con 0x7fabc0070c80 2024-09-10T20:59:11.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7fabbcff9640 1 -- 172.21.15.98:0/42870002 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabb4007030 con 0x7fabc0070c80 2024-09-10T20:59:11.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7fabbcff9640 1 -- 172.21.15.98:0/42870002 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4007460 con 0x7fabc0070c80 2024-09-10T20:59:11.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7fabc69c6640 1 -- 172.21.15.98:0/42870002 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc014d310 con 0x7fabc0070c80 2024-09-10T20:59:11.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7fabc69c6640 1 -- 172.21.15.98:0/42870002 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc013e9b0 con 0x7fabc0070c80 2024-09-10T20:59:11.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.010+0000 7f5805fa4640 1 --2- 172.21.15.98:0/2710856648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f8005a60 0x7f57f80509a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f57f000c410 tx=0x7f57f000c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.010+0000 7fd1add9b640 1 --2- 172.21.15.98:0/1688588162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a801c890 0x7fd1a804caf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0007830 tx=0x7fd1a0007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.009+0000 7f307227f640 1 -- 172.21.15.98:0/3741091072 wait complete. 2024-09-10T20:59:11.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.011+0000 7fabc69c6640 1 -- 172.21.15.98:0/42870002 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc00cde20 con 0x7fabc0070c80 2024-09-10T20:59:11.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.011+0000 7f57f67fc640 1 -- 172.21.15.98:0/2710856648 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0017070 con 0x7f57f8005a60 2024-09-10T20:59:11.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.011+0000 7f57f67fc640 1 -- 172.21.15.98:0/2710856648 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57f00035f0 con 0x7f57f8005a60 2024-09-10T20:59:11.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.011+0000 7f57f67fc640 1 -- 172.21.15.98:0/2710856648 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0017070 con 0x7f57f8005a60 2024-09-10T20:59:11.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.012+0000 7fabbcff9640 1 -- 172.21.15.98:0/42870002 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fabb400f040 con 0x7fabc0070c80 2024-09-10T20:59:11.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.011+0000 7fd19effd640 1 -- 172.21.15.98:0/1688588162 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0019030 con 0x7fd1a801c890 2024-09-10T20:59:11.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.011+0000 7fd19effd640 1 -- 172.21.15.98:0/1688588162 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1a0004020 con 0x7fd1a801c890 2024-09-10T20:59:11.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.011+0000 7fd19effd640 1 -- 172.21.15.98:0/1688588162 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0004450 con 0x7fd1a801c890 2024-09-10T20:59:11.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.011+0000 7fd1af59e640 1 -- 172.21.15.98:0/1688588162 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a81752c0 con 0x7fd1a801c890 2024-09-10T20:59:11.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.011+0000 7fd1af59e640 1 -- 172.21.15.98:0/1688588162 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a8175880 con 0x7fd1a801c890 2024-09-10T20:59:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.012+0000 7f5806fa6640 1 -- 172.21.15.98:0/2710856648 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f80dfd50 con 0x7f57f8005a60 2024-09-10T20:59:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.012+0000 7f5806fa6640 1 -- 172.21.15.98:0/2710856648 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f80d7cd0 con 0x7f57f8005a60 2024-09-10T20:59:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.013+0000 7f307227f640 1 Processor -- start 2024-09-10T20:59:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.014+0000 7f307227f640 1 -- start start 2024-09-10T20:59:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.015+0000 7fd19effd640 1 -- 172.21.15.98:0/1688588162 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd1a0026060 con 0x7fd1a801c890 2024-09-10T20:59:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.014+0000 7fabbcff9640 1 --2- 172.21.15.98:0/42870002 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fabac078040 0x7fabac07a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.014+0000 7fabbcff9640 1 -- 172.21.15.98:0/42870002 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fabb409d6e0 con 0x7fabc0070c80 2024-09-10T20:59:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.015+0000 7fd1af59e640 1 -- 172.21.15.98:0/1688588162 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a8175a80 con 0x7fd1a801c890 2024-09-10T20:59:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.015+0000 7f57f67fc640 1 -- 172.21.15.98:0/2710856648 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f57f00037c0 con 0x7f57f8005a60 2024-09-10T20:59:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.015+0000 7f57f67fc640 1 --2- 172.21.15.98:0/2710856648 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f57e4078370 0x7f57e407a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.016+0000 7f5806fa6640 1 -- 172.21.15.98:0/2710856648 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f8035fe0 con 0x7f57f8005a60 2024-09-10T20:59:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.016+0000 7f57f67fc640 1 -- 172.21.15.98:0/2710856648 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f57f00052e0 con 0x7f57f8005a60 2024-09-10T20:59:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.016+0000 7f307227f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3054003a50 0x7f306c0afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.016+0000 7f58057a3640 1 --2- 172.21.15.98:0/2710856648 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f57e4078370 0x7f57e407a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.017+0000 7f306bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3054003a50 0x7f306c0afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.017+0000 7f306bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3054003a50 0x7f306c0afb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55354/0 (socket says 172.21.15.98:55354) 2024-09-10T20:59:11.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.017+0000 7f58057a3640 1 --2- 172.21.15.98:0/2710856648 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f57e4078370 0x7f57e407a830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f57f8006720 tx=0x7f5800066930 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.017+0000 7f306bfff640 1 -- 172.21.15.98:0/19538424 learned_addr learned my addr 172.21.15.98:0/19538424 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:11.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.017+0000 7f307227f640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306c0b0090 0x7f306c14c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.017+0000 7f307227f640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f306c14c600 0x7f306c0d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.017+0000 7f307227f640 1 -- 172.21.15.98:0/19538424 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f306c0b3a70 con 0x7f3054003a50 2024-09-10T20:59:11.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.017+0000 7f306b7fe640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306c0b0090 0x7f306c14c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.017+0000 7f307081c640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f306c14c600 0x7f306c0d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.018+0000 7f307227f640 1 -- 172.21.15.98:0/19538424 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f306c070a80 con 0x7f306c0b0090 2024-09-10T20:59:11.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.018+0000 7f307227f640 1 -- 172.21.15.98:0/19538424 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f306c0b3bf0 con 0x7f306c14c600 2024-09-10T20:59:11.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.018+0000 7f306bfff640 1 -- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f306c14c600 msgr2=0x7f306c0d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.018+0000 7f306bfff640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f306c14c600 0x7f306c0d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.018+0000 7f306bfff640 1 -- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306c0b0090 msgr2=0x7f306c14c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.018+0000 7f306bfff640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306c0b0090 0x7f306c14c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.018+0000 7f306bfff640 1 -- 172.21.15.98:0/19538424 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306c14f710 con 0x7f3054003a50 2024-09-10T20:59:11.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.019+0000 7f306b7fe640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306c0b0090 0x7f306c14c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:59:11.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.019+0000 7f306bfff640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3054003a50 0x7f306c0afb50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3060031040 tx=0x7f3060006440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.016+0000 7fd19effd640 1 --2- 172.21.15.98:0/1688588162 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd190077f70 0x7fd19007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.016+0000 7fd19effd640 1 -- 172.21.15.98:0/1688588162 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd1a009a630 con 0x7fd1a801c890 2024-09-10T20:59:11.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.016+0000 7fd1ae59c640 1 --2- 172.21.15.98:0/1688588162 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd190077f70 0x7fd19007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.019+0000 7fd1ae59c640 1 --2- 172.21.15.98:0/1688588162 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd190077f70 0x7fd19007a430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0052150 tx=0x7fd1b00b8040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.016+0000 7fabbf7fe640 1 --2- 172.21.15.98:0/42870002 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fabac078040 0x7fabac07a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.017+0000 7fabbf7fe640 1 --2- 172.21.15.98:0/42870002 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fabac078040 0x7fabac07a500 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fabc01395a0 tx=0x7fabb000c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.020+0000 7fabbcff9640 1 -- 172.21.15.98:0/42870002 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fabb40658c0 con 0x7fabc0070c80 2024-09-10T20:59:11.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.022+0000 7f57f67fc640 1 -- 172.21.15.98:0/2710856648 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f57f0098e20 con 0x7f57f8005a60 2024-09-10T20:59:11.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.024+0000 7fd19effd640 1 -- 172.21.15.98:0/1688588162 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd1a8175a80 con 0x7fd1a801c890 2024-09-10T20:59:11.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.025+0000 7f3068ff9640 1 -- 172.21.15.98:0/19538424 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306003cc50 con 0x7f3054003a50 2024-09-10T20:59:11.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.025+0000 7f3068ff9640 1 -- 172.21.15.98:0/19538424 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f306003cdf0 con 0x7f3054003a50 2024-09-10T20:59:11.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.025+0000 7f3068ff9640 1 -- 172.21.15.98:0/19538424 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060003850 con 0x7f3054003a50 2024-09-10T20:59:11.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.025+0000 7f307227f640 1 -- 172.21.15.98:0/19538424 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306c14cc30 con 0x7f3054003a50 2024-09-10T20:59:11.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.025+0000 7f307227f640 1 -- 172.21.15.98:0/19538424 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f306c1f3eb0 con 0x7f3054003a50 2024-09-10T20:59:11.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.027+0000 7f30527fc640 1 -- 172.21.15.98:0/19538424 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f306c1f39e0 con 0x7f3054003a50 2024-09-10T20:59:11.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.040+0000 7f3068ff9640 1 -- 172.21.15.98:0/19538424 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3060044070 con 0x7f3054003a50 2024-09-10T20:59:11.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.040+0000 7f3068ff9640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3058078040 0x7f305807a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.041+0000 7f3068ff9640 1 -- 172.21.15.98:0/19538424 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f30600befb0 con 0x7f3054003a50 2024-09-10T20:59:11.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.041+0000 7f3068ff9640 1 -- 172.21.15.98:0/19538424 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f30600bf470 con 0x7f3054003a50 2024-09-10T20:59:11.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.044+0000 7f306b7fe640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3058078040 0x7f305807a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.047+0000 7f306b7fe640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3058078040 0x7f305807a500 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f306c06d5d0 tx=0x7f305c00b3f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.059+0000 7faa94cbd640 1 -- 172.21.15.98:0/3313250518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9015e470 msgr2=0x7faa9006ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.059+0000 7faa94cbd640 1 --2- 172.21.15.98:0/3313250518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9015e470 0x7faa9006ca50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faa80009900 tx=0x7faa8002a460 comp rx=0 tx=0).stop 2024-09-10T20:59:11.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.059+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1465148533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e18008ba0 msgr2=0x7f4e18054d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.059+0000 7f4e26f4b640 1 --2- 172.21.15.98:0/1465148533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e18008ba0 0x7f4e18054d50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4e20069f00 tx=0x7f4e200a5c10 comp rx=0 tx=0).stop 2024-09-10T20:59:11.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.059+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1465148533 shutdown_connections 2024-09-10T20:59:11.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.059+0000 7f4e26f4b640 1 --2- 172.21.15.98:0/1465148533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e18008ba0 0x7f4e18054d50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.059+0000 7f4e26f4b640 1 --2- 172.21.15.98:0/1465148533 >> v2:172.21.15.190:3300/0 conn(0x7f4e18054970 0x7f4e1804fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.059+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1465148533 >> 172.21.15.98:0/1465148533 conn(0x7f4e1801e8d0 msgr2=0x7f4e180556c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.059+0000 7fad0e79b640 1 -- 172.21.15.98:0/1003773567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad0815b1e0 msgr2=0x7fad0815da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.059+0000 7fad0e79b640 1 --2- 172.21.15.98:0/1003773567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad0815b1e0 0x7fad0815da30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7facf4007350 tx=0x7facf402b010 comp rx=0 tx=0).stop 2024-09-10T20:59:11.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.060+0000 7faa94cbd640 1 -- 172.21.15.98:0/3313250518 shutdown_connections 2024-09-10T20:59:11.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.060+0000 7fad0e79b640 1 -- 172.21.15.98:0/1003773567 shutdown_connections 2024-09-10T20:59:11.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.060+0000 7fad0e79b640 1 --2- 172.21.15.98:0/1003773567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad0815b1e0 0x7fad0815da30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.059+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1465148533 shutdown_connections 2024-09-10T20:59:11.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.060+0000 7fad0e79b640 1 -- 172.21.15.98:0/1003773567 >> 172.21.15.98:0/1003773567 conn(0x7fad080b1760 msgr2=0x7fad080b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.060+0000 7fad0e79b640 1 -- 172.21.15.98:0/1003773567 shutdown_connections 2024-09-10T20:59:11.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.060+0000 7fad0e79b640 1 -- 172.21.15.98:0/1003773567 wait complete. 2024-09-10T20:59:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.060+0000 7faa94cbd640 1 --2- 172.21.15.98:0/3313250518 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa90074c90 0x7faa900ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.060+0000 7faa94cbd640 1 --2- 172.21.15.98:0/3313250518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9015e470 0x7faa9006ca50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.060+0000 7faa94cbd640 1 -- 172.21.15.98:0/3313250518 >> 172.21.15.98:0/3313250518 conn(0x7faa90074170 msgr2=0x7faa90074580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.060+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1465148533 wait complete. 2024-09-10T20:59:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.060+0000 7faa94cbd640 1 -- 172.21.15.98:0/3313250518 shutdown_connections 2024-09-10T20:59:11.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.060+0000 7faa94cbd640 1 -- 172.21.15.98:0/3313250518 wait complete. 2024-09-10T20:59:11.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.061+0000 7f3897d9b640 1 -- 172.21.15.98:0/256900817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3890072560 msgr2=0x7f38900b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.061+0000 7f3894b35640 1 -- 172.21.15.98:0/256900817 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880032630 con 0x7f3890072560 2024-09-10T20:59:11.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.061+0000 7f3897d9b640 1 --2- 172.21.15.98:0/256900817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3890072560 0x7f38900b4960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f38900724f0 tx=0x7f388002a780 comp rx=0 tx=0).stop 2024-09-10T20:59:11.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.061+0000 7f3897d9b640 1 -- 172.21.15.98:0/256900817 shutdown_connections 2024-09-10T20:59:11.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.061+0000 7f3897d9b640 1 --2- 172.21.15.98:0/256900817 >> v2:172.21.15.190:3300/0 conn(0x7f38900b30b0 0x7f3890162130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.061+0000 7f3897d9b640 1 --2- 172.21.15.98:0/256900817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3890072560 0x7f38900b4960 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.061+0000 7f3897d9b640 1 -- 172.21.15.98:0/256900817 >> 172.21.15.98:0/256900817 conn(0x7f38900cfaf0 msgr2=0x7f38900cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.062+0000 7f4e26f4b640 1 Processor -- start 2024-09-10T20:59:11.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.062+0000 7f4e26f4b640 1 -- start start 2024-09-10T20:59:11.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.062+0000 7fad0e79b640 1 Processor -- start 2024-09-10T20:59:11.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.062+0000 7fad0e79b640 1 -- start start 2024-09-10T20:59:11.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.062+0000 7f3897d9b640 1 -- 172.21.15.98:0/256900817 shutdown_connections 2024-09-10T20:59:11.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.063+0000 7faa94cbd640 1 Processor -- start 2024-09-10T20:59:11.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.063+0000 7f3897d9b640 1 -- 172.21.15.98:0/256900817 wait complete. 2024-09-10T20:59:11.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.063+0000 7f4e26f4b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e18054970 0x7f4e1817cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.063+0000 7fad0e79b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad0806fe60 0x7fad08138e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7fad0e79b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad0813cf80 0x7fad08072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7f4e26f4b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e18055dd0 0x7f4e180d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7fad0e79b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0815b1e0 0x7fad08072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7f4e25f49640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e18054970 0x7f4e1817cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7fad0e79b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fad08165150 con 0x7fad0806fe60 2024-09-10T20:59:11.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7fad077fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad0813cf80 0x7fad08072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7f4e25f49640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e18054970 0x7f4e1817cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:32940/0 (socket says 172.21.15.98:32940) 2024-09-10T20:59:11.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7fad077fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad0813cf80 0x7fad08072560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:32956/0 (socket says 172.21.15.98:32956) 2024-09-10T20:59:11.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7fad077fe640 1 -- 172.21.15.98:0/2560303514 learned_addr learned my addr 172.21.15.98:0/2560303514 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:11.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7fad0cd38640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0815b1e0 0x7fad08072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7fad0e79b640 1 -- 172.21.15.98:0/2560303514 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fad08164fd0 con 0x7fad0813cf80 2024-09-10T20:59:11.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7fad0e79b640 1 -- 172.21.15.98:0/2560303514 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fad081652d0 con 0x7fad0815b1e0 2024-09-10T20:59:11.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7f3897d9b640 1 Processor -- start 2024-09-10T20:59:11.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7fad07fff640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad0806fe60 0x7fad08138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.064+0000 7f4e25748640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e18055dd0 0x7f4e180d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7f4e25748640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e18055dd0 0x7f4e180d2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55404/0 (socket says 172.21.15.98:55404) 2024-09-10T20:59:11.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7f4e25f49640 1 -- 172.21.15.98:0/1733906949 learned_addr learned my addr 172.21.15.98:0/1733906949 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:11.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.063+0000 7faa94cbd640 1 -- start start 2024-09-10T20:59:11.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7f4e26f4b640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e180d3090 0x7f4e1803a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa94cbd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa90074c90 0x7faa900aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa94cbd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900ccbd0 0x7faa900af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa94cbd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa9015e470 0x7faa902017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa94cbd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faa900cd030 con 0x7faa90074c90 2024-09-10T20:59:11.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa94cbd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faa900afce0 con 0x7faa900ccbd0 2024-09-10T20:59:11.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa94cbd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7faa900cd1b0 con 0x7faa9015e470 2024-09-10T20:59:11.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1733906949 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4e18036ed0 con 0x7f4e18055dd0 2024-09-10T20:59:11.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa8ed9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa9015e470 0x7faa902017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7f4e2674a640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e180d3090 0x7f4e1803a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7fad0cd38640 1 -- 172.21.15.98:0/2560303514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad0813cf80 msgr2=0x7fad08072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7fad0cd38640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad0813cf80 0x7fad08072560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7fad0cd38640 1 -- 172.21.15.98:0/2560303514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad0806fe60 msgr2=0x7fad08138e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1733906949 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e18036d50 con 0x7f4e18054970 2024-09-10T20:59:11.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1733906949 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4e18052aa0 con 0x7f4e180d3090 2024-09-10T20:59:11.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa8dd9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900ccbd0 0x7faa900af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7fad0cd38640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad0806fe60 0x7fad08138e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa8ed9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa9015e470 0x7faa902017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.98:35460/0 (socket says 172.21.15.98:35460) 2024-09-10T20:59:11.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa8ed9d640 1 -- 172.21.15.98:0/212572697 learned_addr learned my addr 172.21.15.98:0/212572697 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:11.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa8e59c640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa90074c90 0x7faa900aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7f3897d9b640 1 -- start start 2024-09-10T20:59:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7fad0cd38640 1 -- 172.21.15.98:0/2560303514 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad080733f0 con 0x7fad0815b1e0 2024-09-10T20:59:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa8ed9d640 1 -- 172.21.15.98:0/212572697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900ccbd0 msgr2=0x7faa900af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7fad07fff640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad0806fe60 0x7fad08138e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:59:11.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa8ed9d640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900ccbd0 0x7faa900af200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa8ed9d640 1 -- 172.21.15.98:0/212572697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa90074c90 msgr2=0x7faa900aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa8ed9d640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa90074c90 0x7faa900aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa8ed9d640 1 -- 172.21.15.98:0/212572697 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa90202560 con 0x7faa9015e470 2024-09-10T20:59:11.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.065+0000 7faa8dd9b640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900ccbd0 0x7faa900af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:11.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7faa8e59c640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa90074c90 0x7faa900aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:11.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7f4e25f49640 1 -- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e180d3090 msgr2=0x7f4e1803a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7f4e25f49640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e180d3090 0x7f4e1803a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7f4e25f49640 1 -- 172.21.15.98:0/1733906949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e18055dd0 msgr2=0x7f4e180d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7f4e25f49640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e18055dd0 0x7f4e180d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7f4e25f49640 1 -- 172.21.15.98:0/1733906949 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e180e2330 con 0x7f4e18054970 2024-09-10T20:59:11.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7faa8ed9d640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa9015e470 0x7faa902017d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faa8800c7b0 tx=0x7faa8800cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7faa7effd640 1 -- 172.21.15.98:0/212572697 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88004b60 con 0x7faa9015e470 2024-09-10T20:59:11.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7f4e25f49640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e18054970 0x7f4e1817cc20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00d780 tx=0x7f4e1c00dc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7fad0cd38640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0815b1e0 0x7fad08072aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7facf8001e50 tx=0x7facf800c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7faa94cbd640 1 -- 172.21.15.98:0/212572697 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa901e7c90 con 0x7faa9015e470 2024-09-10T20:59:11.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7faa94cbd640 1 -- 172.21.15.98:0/212572697 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faa901e8250 con 0x7faa9015e470 2024-09-10T20:59:11.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7f4e167fc640 1 -- 172.21.15.98:0/1733906949 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c017070 con 0x7f4e18054970 2024-09-10T20:59:11.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7f4e167fc640 1 -- 172.21.15.98:0/1733906949 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e1c00de90 con 0x7f4e18054970 2024-09-10T20:59:11.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.066+0000 7f4e167fc640 1 -- 172.21.15.98:0/1733906949 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c010460 con 0x7f4e18054970 2024-09-10T20:59:11.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7fad04ff9640 1 -- 172.21.15.98:0/2560303514 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8015070 con 0x7fad0815b1e0 2024-09-10T20:59:11.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7fad04ff9640 1 -- 172.21.15.98:0/2560303514 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facf80035f0 con 0x7fad0815b1e0 2024-09-10T20:59:11.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7fad04ff9640 1 -- 172.21.15.98:0/2560303514 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8003a40 con 0x7fad0815b1e0 2024-09-10T20:59:11.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7f3897d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3890072560 0x7f389013fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7f3897d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38900b2cd0 0x7f3890160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7f3897d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38900b30b0 0x7f389013bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7f3895b37640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3890072560 0x7f389013fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7f3895b37640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3890072560 0x7f389013fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.98:32976/0 (socket says 172.21.15.98:32976) 2024-09-10T20:59:11.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7f3895b37640 1 -- 172.21.15.98:0/3963926137 learned_addr learned my addr 172.21.15.98:0/3963926137 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:11.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7f4e25748640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e18055dd0 0x7f4e180d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:11.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7f3897d9b640 1 -- 172.21.15.98:0/3963926137 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f38900cdc00 con 0x7f38900b30b0 2024-09-10T20:59:11.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7f3897d9b640 1 -- 172.21.15.98:0/3963926137 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f38900cda80 con 0x7f3890072560 2024-09-10T20:59:11.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7f3897d9b640 1 -- 172.21.15.98:0/3963926137 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f38900cdd80 con 0x7f38900b2cd0 2024-09-10T20:59:11.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7faa7effd640 1 -- 172.21.15.98:0/212572697 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa88004df0 con 0x7faa9015e470 2024-09-10T20:59:11.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7faa7effd640 1 -- 172.21.15.98:0/212572697 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8800aa90 con 0x7faa9015e470 2024-09-10T20:59:11.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7f4e2674a640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e180d3090 0x7f4e1803a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:11.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1733906949 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1817d220 con 0x7f4e18054970 2024-09-10T20:59:11.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.067+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1733906949 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e1817d7e0 con 0x7f4e18054970 2024-09-10T20:59:11.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.068+0000 7f3896338640 1 --2- 172.21.15.98:0/3963926137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38900b30b0 0x7f389013bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.068+0000 7f3895b37640 1 -- 172.21.15.98:0/3963926137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38900b2cd0 msgr2=0x7f3890160c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T20:59:11.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.068+0000 7f3895b37640 1 --2- 172.21.15.98:0/3963926137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38900b2cd0 0x7f3890160c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.068+0000 7f3895b37640 1 -- 172.21.15.98:0/3963926137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38900b30b0 msgr2=0x7f389013bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.068+0000 7f3895b37640 1 --2- 172.21.15.98:0/3963926137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38900b30b0 0x7f389013bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.068+0000 7fad0e79b640 1 -- 172.21.15.98:0/2560303514 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad08073940 con 0x7fad0815b1e0 2024-09-10T20:59:11.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.068+0000 7fad0e79b640 1 -- 172.21.15.98:0/2560303514 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fad080a9520 con 0x7fad0815b1e0 2024-09-10T20:59:11.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.068+0000 7f3895b37640 1 -- 172.21.15.98:0/3963926137 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389015f220 con 0x7f3890072560 2024-09-10T20:59:11.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.069+0000 7f3895b37640 1 --2- 172.21.15.98:0/3963926137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3890072560 0x7f389013fe40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f388400c430 tx=0x7f388400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.069+0000 7f388e7fc640 1 -- 172.21.15.98:0/3963926137 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884006890 con 0x7f3890072560 2024-09-10T20:59:11.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.069+0000 7faa94cbd640 1 -- 172.21.15.98:0/212572697 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa74005290 con 0x7faa9015e470 2024-09-10T20:59:11.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.069+0000 7fad0e79b640 1 -- 172.21.15.98:0/2560303514 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0806f750 con 0x7fad0815b1e0 2024-09-10T20:59:11.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.069+0000 7f3896338640 1 --2- 172.21.15.98:0/3963926137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38900b30b0 0x7f389013bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:11.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.070+0000 7faa7effd640 1 -- 172.21.15.98:0/212572697 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7faa88023060 con 0x7faa9015e470 2024-09-10T20:59:11.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.071+0000 7faa7effd640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7faa70078040 0x7faa7007a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.070+0000 7f388e7fc640 1 -- 172.21.15.98:0/3963926137 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3884006b20 con 0x7f3890072560 2024-09-10T20:59:11.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.070+0000 7f3897d9b640 1 -- 172.21.15.98:0/3963926137 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3890144d20 con 0x7f3890072560 2024-09-10T20:59:11.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.070+0000 7f388e7fc640 1 -- 172.21.15.98:0/3963926137 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884007020 con 0x7f3890072560 2024-09-10T20:59:11.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.070+0000 7f3897d9b640 1 -- 172.21.15.98:0/3963926137 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f389013ce00 con 0x7f3890072560 2024-09-10T20:59:11.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.069+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1733906949 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e18052f20 con 0x7f4e18054970 2024-09-10T20:59:11.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.071+0000 7faa7effd640 1 -- 172.21.15.98:0/212572697 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faa8809a0a0 con 0x7faa9015e470 2024-09-10T20:59:11.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.072+0000 7f4e167fc640 1 -- 172.21.15.98:0/1733906949 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4e1c010600 con 0x7f4e18054970 2024-09-10T20:59:11.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.072+0000 7f3897d9b640 1 -- 172.21.15.98:0/3963926137 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38900b4960 con 0x7f3890072560 2024-09-10T20:59:11.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.072+0000 7f388e7fc640 1 -- 172.21.15.98:0/3963926137 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f38840071c0 con 0x7f3890072560 2024-09-10T20:59:11.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.073+0000 7f4e167fc640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4e04078070 0x7f4e0407a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.073+0000 7f4e167fc640 1 -- 172.21.15.98:0/1733906949 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4e1c09a550 con 0x7f4e18054970 2024-09-10T20:59:11.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.072+0000 7faa8e59c640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7faa70078040 0x7faa7007a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.073+0000 7f4e25748640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4e04078070 0x7f4e0407a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.073+0000 7fad04ff9640 1 -- 172.21.15.98:0/2560303514 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7facf8022060 con 0x7fad0815b1e0 2024-09-10T20:59:11.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.073+0000 7f388e7fc640 1 --2- 172.21.15.98:0/3963926137 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f387c078040 0x7f387c07a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.073+0000 7f3895336640 1 --2- 172.21.15.98:0/3963926137 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f387c078040 0x7f387c07a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.074+0000 7faa8e59c640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7faa70078040 0x7faa7007a500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faa80033000 tx=0x7faa80004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.074+0000 7f388e7fc640 1 -- 172.21.15.98:0/3963926137 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3884099bd0 con 0x7f3890072560 2024-09-10T20:59:11.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.074+0000 7f4e25748640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4e04078070 0x7f4e0407a530 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4e180381e0 tx=0x7f4e200a6f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.074+0000 7fad04ff9640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7facf0078040 0x7facf007a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.074+0000 7fad04ff9640 1 -- 172.21.15.98:0/2560303514 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7facf809a410 con 0x7fad0815b1e0 2024-09-10T20:59:11.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.074+0000 7fad07fff640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7facf0078040 0x7facf007a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.074+0000 7f3895336640 1 --2- 172.21.15.98:0/3963926137 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f387c078040 0x7f387c07a500 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3880033000 tx=0x7f3880004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.074+0000 7fad07fff640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7facf0078040 0x7facf007a500 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fad0000a7f0 tx=0x7fad0000d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.075+0000 7fad04ff9640 1 -- 172.21.15.98:0/2560303514 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7facf8062c40 con 0x7fad0815b1e0 2024-09-10T20:59:11.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.077+0000 7f4e167fc640 1 -- 172.21.15.98:0/1733906949 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4e1c062cf0 con 0x7f4e18054970 2024-09-10T20:59:11.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.080+0000 7faa7effd640 1 -- 172.21.15.98:0/212572697 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7faa880628d0 con 0x7faa9015e470 2024-09-10T20:59:11.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.084+0000 7f388e7fc640 1 -- 172.21.15.98:0/3963926137 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3884062370 con 0x7f3890072560 2024-09-10T20:59:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.423+0000 7fabc69c6640 1 -- 172.21.15.98:0/42870002 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fabc0143570 con 0x7fabc0070c80 2024-09-10T20:59:11.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.424+0000 7fabbcff9640 1 -- 172.21.15.98:0/42870002 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fabb406a2f0 con 0x7fabc0070c80 2024-09-10T20:59:11.428 INFO:teuthology.orchestra.run.smithi098.stdout:55834574858 2024-09-10T20:59:11.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.427+0000 7faba67fc640 1 -- 172.21.15.98:0/42870002 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fabac078040 msgr2=0x7fabac07a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.428+0000 7faba67fc640 1 --2- 172.21.15.98:0/42870002 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fabac078040 0x7fabac07a500 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fabc01395a0 tx=0x7fabb000c040 comp rx=0 tx=0).stop 2024-09-10T20:59:11.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.428+0000 7faba67fc640 1 -- 172.21.15.98:0/42870002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc0070c80 msgr2=0x7fabc013bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.428+0000 7faba67fc640 1 --2- 172.21.15.98:0/42870002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc0070c80 0x7fabc013bd20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fabb4007fb0 tx=0x7fabb400eae0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.428+0000 7faba67fc640 1 -- 172.21.15.98:0/42870002 shutdown_connections 2024-09-10T20:59:11.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.428+0000 7faba67fc640 1 --2- 172.21.15.98:0/42870002 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fabac078040 0x7fabac07a500 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.428+0000 7f5806fa6640 1 -- 172.21.15.98:0/2710856648 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f57f80eab00 con 0x7f57f8005a60 2024-09-10T20:59:11.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.428+0000 7faba67fc640 1 --2- 172.21.15.98:0/42870002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabc015db10 0x7fabc01f3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.428+0000 7faba67fc640 1 --2- 172.21.15.98:0/42870002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fabc00cd9f0 0x7fabc013f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.428+0000 7faba67fc640 1 --2- 172.21.15.98:0/42870002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fabc0070c80 0x7fabc013bd20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.428+0000 7faba67fc640 1 -- 172.21.15.98:0/42870002 >> 172.21.15.98:0/42870002 conn(0x7fabc00cfaf0 msgr2=0x7fabc006fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.428+0000 7faba67fc640 1 -- 172.21.15.98:0/42870002 shutdown_connections 2024-09-10T20:59:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.428+0000 7faba67fc640 1 -- 172.21.15.98:0/42870002 wait complete. 2024-09-10T20:59:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.429+0000 7f57f67fc640 1 -- 172.21.15.98:0/2710856648 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f57f0060ee0 con 0x7f57f8005a60 2024-09-10T20:59:11.434 INFO:teuthology.orchestra.run.smithi098.stdout:60129542154 2024-09-10T20:59:11.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.430+0000 7fd1af59e640 1 -- 172.21.15.98:0/1688588162 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd1a8035fe0 con 0x7fd1a801c890 2024-09-10T20:59:11.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.431+0000 7fd19effd640 1 -- 172.21.15.98:0/1688588162 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd1a0062700 con 0x7fd1a801c890 2024-09-10T20:59:11.435 INFO:teuthology.orchestra.run.smithi098.stdout:64424509450 2024-09-10T20:59:11.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.433+0000 7f57e3fff640 1 -- 172.21.15.98:0/2710856648 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f57e4078370 msgr2=0x7f57e407a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.433+0000 7f57e3fff640 1 --2- 172.21.15.98:0/2710856648 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f57e4078370 0x7f57e407a830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f57f8006720 tx=0x7f5800066930 comp rx=0 tx=0).stop 2024-09-10T20:59:11.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.433+0000 7f57e3fff640 1 -- 172.21.15.98:0/2710856648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f8005a60 msgr2=0x7f57f80509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.433+0000 7f57e3fff640 1 --2- 172.21.15.98:0/2710856648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f8005a60 0x7f57f80509a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f57f000c410 tx=0x7f57f000c8d0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.434+0000 7f57e3fff640 1 -- 172.21.15.98:0/2710856648 shutdown_connections 2024-09-10T20:59:11.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.434+0000 7f57e3fff640 1 --2- 172.21.15.98:0/2710856648 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f57e4078370 0x7f57e407a830 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.434+0000 7f57e3fff640 1 --2- 172.21.15.98:0/2710856648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57f80366f0 0x7f57f80e0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.434+0000 7f57e3fff640 1 --2- 172.21.15.98:0/2710856648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57f800ba00 0x7f57f80df2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.434+0000 7f57e3fff640 1 --2- 172.21.15.98:0/2710856648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f8005a60 0x7f57f80509a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.434+0000 7f57e3fff640 1 -- 172.21.15.98:0/2710856648 >> 172.21.15.98:0/2710856648 conn(0x7f57f8054560 msgr2=0x7f57f8053380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.434+0000 7f57e3fff640 1 -- 172.21.15.98:0/2710856648 shutdown_connections 2024-09-10T20:59:11.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.434+0000 7f57e3fff640 1 -- 172.21.15.98:0/2710856648 wait complete. 2024-09-10T20:59:11.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.434+0000 7fd19cff9640 1 -- 172.21.15.98:0/1688588162 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd190077f70 msgr2=0x7fd19007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.434+0000 7fd19cff9640 1 --2- 172.21.15.98:0/1688588162 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd190077f70 0x7fd19007a430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0052150 tx=0x7fd1b00b8040 comp rx=0 tx=0).stop 2024-09-10T20:59:11.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.434+0000 7fd19cff9640 1 -- 172.21.15.98:0/1688588162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a801c890 msgr2=0x7fd1a804caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.434+0000 7fd19cff9640 1 --2- 172.21.15.98:0/1688588162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a801c890 0x7fd1a804caf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0007830 tx=0x7fd1a0007cf0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.435+0000 7fd19cff9640 1 -- 172.21.15.98:0/1688588162 shutdown_connections 2024-09-10T20:59:11.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.435+0000 7fd19cff9640 1 --2- 172.21.15.98:0/1688588162 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd190077f70 0x7fd19007a430 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.435+0000 7fd19cff9640 1 --2- 172.21.15.98:0/1688588162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1a8036520 0x7fd1a804d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.435+0000 7fd19cff9640 1 --2- 172.21.15.98:0/1688588162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1a801c890 0x7fd1a804caf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.435+0000 7fd19cff9640 1 --2- 172.21.15.98:0/1688588162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a8005a60 0x7fd1a80e1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.435+0000 7fd19cff9640 1 -- 172.21.15.98:0/1688588162 >> 172.21.15.98:0/1688588162 conn(0x7fd1a8054560 msgr2=0x7fd1a8052b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.435+0000 7fd19cff9640 1 -- 172.21.15.98:0/1688588162 shutdown_connections 2024-09-10T20:59:11.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.435+0000 7fd19cff9640 1 -- 172.21.15.98:0/1688588162 wait complete. 2024-09-10T20:59:11.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2024-09-10T20:59:11.451 DEBUG:teuthology.parallel:result is None 2024-09-10T20:59:11.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542154 for osd.0 2024-09-10T20:59:11.456 DEBUG:teuthology.parallel:result is None 2024-09-10T20:59:11.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.7 2024-09-10T20:59:11.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.453+0000 7f30527fc640 1 -- 172.21.15.98:0/19538424 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f306c0b4180 con 0x7f3054003a50 2024-09-10T20:59:11.458 DEBUG:teuthology.parallel:result is None 2024-09-10T20:59:11.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.454+0000 7f3068ff9640 1 -- 172.21.15.98:0/19538424 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f306008ee20 con 0x7f3054003a50 2024-09-10T20:59:11.458 INFO:teuthology.orchestra.run.smithi098.stdout:64424509450 2024-09-10T20:59:11.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.457+0000 7f30527fc640 1 -- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3058078040 msgr2=0x7f305807a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.457+0000 7f30527fc640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3058078040 0x7f305807a500 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f306c06d5d0 tx=0x7f305c00b3f0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.457+0000 7f30527fc640 1 -- 172.21.15.98:0/19538424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3054003a50 msgr2=0x7f306c0afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.457+0000 7f30527fc640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3054003a50 0x7f306c0afb50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3060031040 tx=0x7f3060006440 comp rx=0 tx=0).stop 2024-09-10T20:59:11.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.457+0000 7f30527fc640 1 -- 172.21.15.98:0/19538424 shutdown_connections 2024-09-10T20:59:11.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.457+0000 7f30527fc640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3058078040 0x7f305807a500 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.457+0000 7f30527fc640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f306c14c600 0x7f306c0d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.457+0000 7f30527fc640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f306c0b0090 0x7f306c14c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.457+0000 7f30527fc640 1 --2- 172.21.15.98:0/19538424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3054003a50 0x7f306c0afb50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.457+0000 7f30527fc640 1 -- 172.21.15.98:0/19538424 >> 172.21.15.98:0/19538424 conn(0x7f306c0b2770 msgr2=0x7f306c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.457+0000 7f30527fc640 1 -- 172.21.15.98:0/19538424 shutdown_connections 2024-09-10T20:59:11.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.457+0000 7f30527fc640 1 -- 172.21.15.98:0/19538424 wait complete. 2024-09-10T20:59:11.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.470+0000 7fad0e79b640 1 -- 172.21.15.98:0/2560303514 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fad080a96f0 con 0x7fad0815b1e0 2024-09-10T20:59:11.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.470+0000 7fad04ff9640 1 -- 172.21.15.98:0/2560303514 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7facf8069030 con 0x7fad0815b1e0 2024-09-10T20:59:11.474 INFO:teuthology.orchestra.run.smithi098.stdout:60129542154 2024-09-10T20:59:11.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.473+0000 7fad0e79b640 1 -- 172.21.15.98:0/2560303514 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7facf0078040 msgr2=0x7facf007a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.473+0000 7fad0e79b640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7facf0078040 0x7facf007a500 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fad0000a7f0 tx=0x7fad0000d040 comp rx=0 tx=0).stop 2024-09-10T20:59:11.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.473+0000 7fad0e79b640 1 -- 172.21.15.98:0/2560303514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0815b1e0 msgr2=0x7fad08072aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.473+0000 7fad0e79b640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0815b1e0 0x7fad08072aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7facf8001e50 tx=0x7facf800c990 comp rx=0 tx=0).stop 2024-09-10T20:59:11.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.473+0000 7fad0e79b640 1 -- 172.21.15.98:0/2560303514 shutdown_connections 2024-09-10T20:59:11.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.473+0000 7fad0e79b640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7facf0078040 0x7facf007a500 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.473+0000 7fad0e79b640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0815b1e0 0x7fad08072aa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.473+0000 7fad0e79b640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad0813cf80 0x7fad08072560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.473+0000 7fad0e79b640 1 --2- 172.21.15.98:0/2560303514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad0806fe60 0x7fad08138e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.473+0000 7fad0e79b640 1 -- 172.21.15.98:0/2560303514 >> 172.21.15.98:0/2560303514 conn(0x7fad080b1760 msgr2=0x7fad08074220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.4 2024-09-10T20:59:11.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.473+0000 7fad0e79b640 1 -- 172.21.15.98:0/2560303514 shutdown_connections 2024-09-10T20:59:11.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.473+0000 7fad0e79b640 1 -- 172.21.15.98:0/2560303514 wait complete. 2024-09-10T20:59:11.479 DEBUG:teuthology.parallel:result is None 2024-09-10T20:59:11.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.475+0000 7f3897d9b640 1 -- 172.21.15.98:0/3963926137 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3890146c10 con 0x7f3890072560 2024-09-10T20:59:11.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.475+0000 7f388e7fc640 1 -- 172.21.15.98:0/3963926137 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3884061c70 con 0x7f3890072560 2024-09-10T20:59:11.479 INFO:teuthology.orchestra.run.smithi098.stdout:60129542154 2024-09-10T20:59:11.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.476+0000 7faa94cbd640 1 -- 172.21.15.98:0/212572697 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7faa74005580 con 0x7faa9015e470 2024-09-10T20:59:11.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.477+0000 7faa7effd640 1 -- 172.21.15.98:0/212572697 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faa880621d0 con 0x7faa9015e470 2024-09-10T20:59:11.481 INFO:teuthology.orchestra.run.smithi098.stdout:60129542154 2024-09-10T20:59:11.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.478+0000 7f3897d9b640 1 -- 172.21.15.98:0/3963926137 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f387c078040 msgr2=0x7f387c07a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.478+0000 7f3897d9b640 1 --2- 172.21.15.98:0/3963926137 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f387c078040 0x7f387c07a500 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3880033000 tx=0x7f3880004ab0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.478+0000 7f3897d9b640 1 -- 172.21.15.98:0/3963926137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3890072560 msgr2=0x7f389013fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.478+0000 7f3897d9b640 1 --2- 172.21.15.98:0/3963926137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3890072560 0x7f389013fe40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f388400c430 tx=0x7f388400c8f0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.478+0000 7f3897d9b640 1 -- 172.21.15.98:0/3963926137 shutdown_connections 2024-09-10T20:59:11.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.478+0000 7f3897d9b640 1 --2- 172.21.15.98:0/3963926137 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f387c078040 0x7f387c07a500 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.478+0000 7f3897d9b640 1 --2- 172.21.15.98:0/3963926137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38900b30b0 0x7f389013bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.478+0000 7f3897d9b640 1 --2- 172.21.15.98:0/3963926137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38900b2cd0 0x7f3890160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.478+0000 7f3897d9b640 1 --2- 172.21.15.98:0/3963926137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3890072560 0x7f389013fe40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.478+0000 7f3897d9b640 1 -- 172.21.15.98:0/3963926137 >> 172.21.15.98:0/3963926137 conn(0x7f38900cfaf0 msgr2=0x7f38901619b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.478+0000 7f3897d9b640 1 -- 172.21.15.98:0/3963926137 shutdown_connections 2024-09-10T20:59:11.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.478+0000 7f3897d9b640 1 -- 172.21.15.98:0/3963926137 wait complete. 2024-09-10T20:59:11.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.480+0000 7faa94cbd640 1 -- 172.21.15.98:0/212572697 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7faa70078040 msgr2=0x7faa7007a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.480+0000 7faa94cbd640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7faa70078040 0x7faa7007a500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faa80033000 tx=0x7faa80004730 comp rx=0 tx=0).stop 2024-09-10T20:59:11.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.480+0000 7faa94cbd640 1 -- 172.21.15.98:0/212572697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa9015e470 msgr2=0x7faa902017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.480+0000 7faa94cbd640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa9015e470 0x7faa902017d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faa8800c7b0 tx=0x7faa8800cc70 comp rx=0 tx=0).stop 2024-09-10T20:59:11.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.480+0000 7faa94cbd640 1 -- 172.21.15.98:0/212572697 shutdown_connections 2024-09-10T20:59:11.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.480+0000 7faa94cbd640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7faa70078040 0x7faa7007a500 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.480+0000 7faa94cbd640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa9015e470 0x7faa902017d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.480+0000 7faa94cbd640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa900ccbd0 0x7faa900af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.480+0000 7faa94cbd640 1 --2- 172.21.15.98:0/212572697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa90074c90 0x7faa900aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.480+0000 7faa94cbd640 1 -- 172.21.15.98:0/212572697 >> 172.21.15.98:0/212572697 conn(0x7faa90074170 msgr2=0x7faa900cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.480+0000 7faa94cbd640 1 -- 172.21.15.98:0/212572697 shutdown_connections 2024-09-10T20:59:11.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.480+0000 7faa94cbd640 1 -- 172.21.15.98:0/212572697 wait complete. 2024-09-10T20:59:11.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.482+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1733906949 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4e180337a0 con 0x7f4e18054970 2024-09-10T20:59:11.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.483+0000 7f4e167fc640 1 -- 172.21.15.98:0/1733906949 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4e1c0625f0 con 0x7f4e18054970 2024-09-10T20:59:11.487 INFO:teuthology.orchestra.run.smithi098.stdout:64424509450 2024-09-10T20:59:11.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.486+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4e04078070 msgr2=0x7f4e0407a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.486+0000 7f4e26f4b640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4e04078070 0x7f4e0407a530 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4e180381e0 tx=0x7f4e200a6f90 comp rx=0 tx=0).stop 2024-09-10T20:59:11.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.486+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e18054970 msgr2=0x7f4e1817cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.486+0000 7f4e26f4b640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e18054970 0x7f4e1817cc20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00d780 tx=0x7f4e1c00dc40 comp rx=0 tx=0).stop 2024-09-10T20:59:11.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.486+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1733906949 shutdown_connections 2024-09-10T20:59:11.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.486+0000 7f4e26f4b640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4e04078070 0x7f4e0407a530 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.486+0000 7f4e26f4b640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e180d3090 0x7f4e1803a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.486+0000 7f4e26f4b640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e18055dd0 0x7f4e180d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.486+0000 7f4e26f4b640 1 --2- 172.21.15.98:0/1733906949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e18054970 0x7f4e1817cc20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.486+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1733906949 >> 172.21.15.98:0/1733906949 conn(0x7f4e1801e8d0 msgr2=0x7f4e1800b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.486+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1733906949 shutdown_connections 2024-09-10T20:59:11.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.486+0000 7f4e26f4b640 1 -- 172.21.15.98:0/1733906949 wait complete. 2024-09-10T20:59:11.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542154 for osd.2 2024-09-10T20:59:11.492 DEBUG:teuthology.parallel:result is None 2024-09-10T20:59:11.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542154 for osd.3 2024-09-10T20:59:11.496 DEBUG:teuthology.parallel:result is None 2024-09-10T20:59:11.496 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542154 for osd.1 2024-09-10T20:59:11.496 DEBUG:teuthology.parallel:result is None 2024-09-10T20:59:11.501 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.6 2024-09-10T20:59:11.502 DEBUG:teuthology.parallel:result is None 2024-09-10T20:59:11.502 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T20:59:11.502 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:59:11.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.639+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3664816494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b50003a00 msgr2=0x7f9b50003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.639+0000 7f9b6beb8640 1 --2- 172.21.15.98:0/3664816494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b50003a00 0x7f9b50003e20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000c260 tx=0x7f9b50005830 comp rx=0 tx=0).stop 2024-09-10T20:59:11.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.639+0000 7f9b6a455640 1 -- 172.21.15.98:0/3664816494 reap_dead start 2024-09-10T20:59:11.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.639+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3664816494 shutdown_connections 2024-09-10T20:59:11.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.639+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3664816494 >> 172.21.15.98:0/3664816494 conn(0x7f9b64074150 msgr2=0x7f9b64074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.639+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3664816494 shutdown_connections 2024-09-10T20:59:11.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.639+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3664816494 wait complete. 2024-09-10T20:59:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.641+0000 7f9b6beb8640 1 Processor -- start 2024-09-10T20:59:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.641+0000 7f9b6beb8640 1 -- start start 2024-09-10T20:59:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.641+0000 7f9b6beb8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b50003a00 0x7f9b640c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.641+0000 7f9b6beb8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b640c7690 0x7f9b64159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.641+0000 7f9b6beb8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b640ae310 0x7f9b640ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.641+0000 7f9b6beb8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9b640d0640 con 0x7f9b50003a00 2024-09-10T20:59:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.641+0000 7f9b6beb8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9b640d04c0 con 0x7f9b640ae310 2024-09-10T20:59:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.641+0000 7f9b6beb8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9b640b0c70 con 0x7f9b640c7690 2024-09-10T20:59:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.641+0000 7f9b69c54640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b50003a00 0x7f9b640c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.641+0000 7f9b69c54640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b50003a00 0x7f9b640c47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55444/0 (socket says 172.21.15.98:55444) 2024-09-10T20:59:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.641+0000 7f9b69c54640 1 -- 172.21.15.98:0/3628602002 learned_addr learned my addr 172.21.15.98:0/3628602002 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.641+0000 7f9b6a455640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b640ae310 0x7f9b640ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.642+0000 7f9b69453640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b640c7690 0x7f9b64159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.642+0000 7f9b69c54640 1 -- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b640c7690 msgr2=0x7f9b64159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.642+0000 7f9b69c54640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b640c7690 0x7f9b64159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.642+0000 7f9b69c54640 1 -- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b640ae310 msgr2=0x7f9b640ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.642+0000 7f9b69c54640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b640ae310 0x7f9b640ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.642+0000 7f9b69c54640 1 -- 172.21.15.98:0/3628602002 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b640c7b50 con 0x7f9b50003a00 2024-09-10T20:59:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.642+0000 7f9b6a455640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b640ae310 0x7f9b640ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.642+0000 7f9b69c54640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b50003a00 0x7f9b640c47b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b6002b0f0 tx=0x7f9b60001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.642+0000 7f9b5a7fc640 1 -- 172.21.15.98:0/3628602002 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b60031040 con 0x7f9b50003a00 2024-09-10T20:59:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.642+0000 7f9b5a7fc640 1 -- 172.21.15.98:0/3628602002 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b600035f0 con 0x7f9b50003a00 2024-09-10T20:59:11.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.642+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3628602002 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b640c7e30 con 0x7f9b50003a00 2024-09-10T20:59:11.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.642+0000 7f9b5a7fc640 1 -- 172.21.15.98:0/3628602002 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b600039e0 con 0x7f9b50003a00 2024-09-10T20:59:11.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.642+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3628602002 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b640c83f0 con 0x7f9b50003a00 2024-09-10T20:59:11.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.643+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3628602002 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b640ca360 con 0x7f9b50003a00 2024-09-10T20:59:11.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.643+0000 7f9b5a7fc640 1 -- 172.21.15.98:0/3628602002 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9b60048030 con 0x7f9b50003a00 2024-09-10T20:59:11.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.644+0000 7f9b5a7fc640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9b48077f70 0x7f9b4807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.644+0000 7f9b5a7fc640 1 -- 172.21.15.98:0/3628602002 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9b600b9f50 con 0x7f9b50003a00 2024-09-10T20:59:11.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.644+0000 7f9b69453640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9b48077f70 0x7f9b4807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.645+0000 7f9b69453640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9b48077f70 0x7f9b4807a430 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9b640c3cd0 tx=0x7f9b5400b260 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.648+0000 7f9b5a7fc640 1 -- 172.21.15.98:0/3628602002 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9b600827f0 con 0x7f9b50003a00 2024-09-10T20:59:11.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.818+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3628602002 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b640b0730 con 0x7f9b48077f70 2024-09-10T20:59:11.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.824+0000 7f9b5a7fc640 1 -- 172.21.15.98:0/3628602002 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f9b640b0730 con 0x7f9b48077f70 2024-09-10T20:59:11.831 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:11.833 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-10T20:59:11.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.827+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9b48077f70 msgr2=0x7f9b4807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.827+0000 7f9b6beb8640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9b48077f70 0x7f9b4807a430 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9b640c3cd0 tx=0x7f9b5400b260 comp rx=0 tx=0).stop 2024-09-10T20:59:11.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.827+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3628602002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b50003a00 msgr2=0x7f9b640c47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.827+0000 7f9b6beb8640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b50003a00 0x7f9b640c47b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b6002b0f0 tx=0x7f9b60001d20 comp rx=0 tx=0).stop 2024-09-10T20:59:11.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.827+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3628602002 shutdown_connections 2024-09-10T20:59:11.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.827+0000 7f9b6beb8640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9b48077f70 0x7f9b4807a430 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.827+0000 7f9b6beb8640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b640ae310 0x7f9b640ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.827+0000 7f9b6beb8640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b640c7690 0x7f9b64159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.827+0000 7f9b6beb8640 1 --2- 172.21.15.98:0/3628602002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b50003a00 0x7f9b640c47b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.827+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3628602002 >> 172.21.15.98:0/3628602002 conn(0x7f9b64074150 msgr2=0x7f9b640b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.828+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3628602002 shutdown_connections 2024-09-10T20:59:11.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.828+0000 7f9b6beb8640 1 -- 172.21.15.98:0/3628602002 wait complete. 2024-09-10T20:59:11.842 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-09-10T20:59:10.912643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3317760,"data_stored":2011640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003089"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904144+0000","last_change":"2024-09-10T20:58:46.222874+0000","last_active":"2024-09-10T20:58:53.904144+0000","last_peered":"2024-09-10T20:58:53.904144+0000","last_clean":"2024-09-10T20:58:53.904144+0000","last_became_active":"2024-09-10T20:58:46.222585+0000","last_became_peered":"2024-09-10T20:58:46.222585+0000","last_unstale":"2024-09-10T20:58:53.904144+0000","last_undegraded":"2024-09-10T20:58:53.904144+0000","last_fullsized":"2024-09-10T20:58:53.904144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:12.557183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258747+0000","last_change":"2024-09-10T20:58:47.236553+0000","last_active":"2024-09-10T20:58:52.258747+0000","last_peered":"2024-09-10T20:58:52.258747+0000","last_clean":"2024-09-10T20:58:52.258747+0000","last_became_active":"2024-09-10T20:58:47.236267+0000","last_became_peered":"2024-09-10T20:58:47.236267+0000","last_unstale":"2024-09-10T20:58:52.258747+0000","last_undegraded":"2024-09-10T20:58:52.258747+0000","last_fullsized":"2024-09-10T20:58:52.258747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:54.301892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258094+0000","last_change":"2024-09-10T20:58:51.557088+0000","last_active":"2024-09-10T20:58:52.258094+0000","last_peered":"2024-09-10T20:58:52.258094+0000","last_clean":"2024-09-10T20:58:52.258094+0000","last_became_active":"2024-09-10T20:58:51.556571+0000","last_became_peered":"2024-09-10T20:58:51.556571+0000","last_unstale":"2024-09-10T20:58:52.258094+0000","last_undegraded":"2024-09-10T20:58:52.258094+0000","last_fullsized":"2024-09-10T20:58:52.258094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:32.723152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258861+0000","last_change":"2024-09-10T20:58:46.221311+0000","last_active":"2024-09-10T20:58:52.258861+0000","last_peered":"2024-09-10T20:58:52.258861+0000","last_clean":"2024-09-10T20:58:52.258861+0000","last_became_active":"2024-09-10T20:58:46.221053+0000","last_became_peered":"2024-09-10T20:58:46.221053+0000","last_unstale":"2024-09-10T20:58:52.258861+0000","last_undegraded":"2024-09-10T20:58:52.258861+0000","last_fullsized":"2024-09-10T20:58:52.258861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:02.951532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903969+0000","last_change":"2024-09-10T20:58:47.246604+0000","last_active":"2024-09-10T20:58:53.903969+0000","last_peered":"2024-09-10T20:58:53.903969+0000","last_clean":"2024-09-10T20:58:53.903969+0000","last_became_active":"2024-09-10T20:58:47.244966+0000","last_became_peered":"2024-09-10T20:58:47.244966+0000","last_unstale":"2024-09-10T20:58:53.903969+0000","last_undegraded":"2024-09-10T20:58:53.903969+0000","last_fullsized":"2024-09-10T20:58:53.903969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:54.301892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255041+0000","last_change":"2024-09-10T20:58:51.275679+0000","last_active":"2024-09-10T20:58:52.255041+0000","last_peered":"2024-09-10T20:58:52.255041+0000","last_clean":"2024-09-10T20:58:52.255041+0000","last_became_active":"2024-09-10T20:58:51.275227+0000","last_became_peered":"2024-09-10T20:58:51.275227+0000","last_unstale":"2024-09-10T20:58:52.255041+0000","last_undegraded":"2024-09-10T20:58:52.255041+0000","last_fullsized":"2024-09-10T20:58:52.255041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:07.240517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905109+0000","last_change":"2024-09-10T20:58:46.223303+0000","last_active":"2024-09-10T20:58:53.905109+0000","last_peered":"2024-09-10T20:58:53.905109+0000","last_clean":"2024-09-10T20:58:53.905109+0000","last_became_active":"2024-09-10T20:58:46.222924+0000","last_became_peered":"2024-09-10T20:58:46.222924+0000","last_unstale":"2024-09-10T20:58:53.905109+0000","last_undegraded":"2024-09-10T20:58:53.905109+0000","last_fullsized":"2024-09-10T20:58:53.905109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:37.987408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255679+0000","last_change":"2024-09-10T20:58:47.231063+0000","last_active":"2024-09-10T20:58:52.255679+0000","last_peered":"2024-09-10T20:58:52.255679+0000","last_clean":"2024-09-10T20:58:52.255679+0000","last_became_active":"2024-09-10T20:58:47.230764+0000","last_became_peered":"2024-09-10T20:58:47.230764+0000","last_unstale":"2024-09-10T20:58:52.255679+0000","last_undegraded":"2024-09-10T20:58:52.255679+0000","last_fullsized":"2024-09-10T20:58:52.255679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:17.763154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256830+0000","last_change":"2024-09-10T20:58:51.275019+0000","last_active":"2024-09-10T20:58:52.256830+0000","last_peered":"2024-09-10T20:58:52.256830+0000","last_clean":"2024-09-10T20:58:52.256830+0000","last_became_active":"2024-09-10T20:58:51.274298+0000","last_became_peered":"2024-09-10T20:58:51.274298+0000","last_unstale":"2024-09-10T20:58:52.256830+0000","last_undegraded":"2024-09-10T20:58:52.256830+0000","last_fullsized":"2024-09-10T20:58:52.256830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.486482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904772+0000","last_change":"2024-09-10T20:58:46.231109+0000","last_active":"2024-09-10T20:58:53.904772+0000","last_peered":"2024-09-10T20:58:53.904772+0000","last_clean":"2024-09-10T20:58:53.904772+0000","last_became_active":"2024-09-10T20:58:46.230798+0000","last_became_peered":"2024-09-10T20:58:46.230798+0000","last_unstale":"2024-09-10T20:58:53.904772+0000","last_undegraded":"2024-09-10T20:58:53.904772+0000","last_fullsized":"2024-09-10T20:58:53.904772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:31.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256327+0000","last_change":"2024-09-10T20:58:47.230225+0000","last_active":"2024-09-10T20:58:52.256327+0000","last_peered":"2024-09-10T20:58:52.256327+0000","last_clean":"2024-09-10T20:58:52.256327+0000","last_became_active":"2024-09-10T20:58:47.229943+0000","last_became_peered":"2024-09-10T20:58:47.229943+0000","last_unstale":"2024-09-10T20:58:52.256327+0000","last_undegraded":"2024-09-10T20:58:52.256327+0000","last_fullsized":"2024-09-10T20:58:52.256327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:45.849686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256190+0000","last_change":"2024-09-10T20:58:51.270577+0000","last_active":"2024-09-10T20:58:52.256190+0000","last_peered":"2024-09-10T20:58:52.256190+0000","last_clean":"2024-09-10T20:58:52.256190+0000","last_became_active":"2024-09-10T20:58:51.270223+0000","last_became_peered":"2024-09-10T20:58:51.270223+0000","last_unstale":"2024-09-10T20:58:52.256190+0000","last_undegraded":"2024-09-10T20:58:52.256190+0000","last_fullsized":"2024-09-10T20:58:52.256190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:45.149805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904189+0000","last_change":"2024-09-10T20:58:47.230068+0000","last_active":"2024-09-10T20:58:53.904189+0000","last_peered":"2024-09-10T20:58:53.904189+0000","last_clean":"2024-09-10T20:58:53.904189+0000","last_became_active":"2024-09-10T20:58:47.229755+0000","last_became_peered":"2024-09-10T20:58:47.229755+0000","last_unstale":"2024-09-10T20:58:53.904189+0000","last_undegraded":"2024-09-10T20:58:53.904189+0000","last_fullsized":"2024-09-10T20:58:53.904189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:04.498018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904966+0000","last_change":"2024-09-10T20:58:46.227149+0000","last_active":"2024-09-10T20:58:53.904966+0000","last_peered":"2024-09-10T20:58:53.904966+0000","last_clean":"2024-09-10T20:58:53.904966+0000","last_became_active":"2024-09-10T20:58:46.226863+0000","last_became_peered":"2024-09-10T20:58:46.226863+0000","last_unstale":"2024-09-10T20:58:53.904966+0000","last_undegraded":"2024-09-10T20:58:53.904966+0000","last_fullsized":"2024-09-10T20:58:53.904966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:28.108849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256521+0000","last_change":"2024-09-10T20:58:51.280288+0000","last_active":"2024-09-10T20:58:52.256521+0000","last_peered":"2024-09-10T20:58:52.256521+0000","last_clean":"2024-09-10T20:58:52.256521+0000","last_became_active":"2024-09-10T20:58:51.279918+0000","last_became_peered":"2024-09-10T20:58:51.279918+0000","last_unstale":"2024-09-10T20:58:52.256521+0000","last_undegraded":"2024-09-10T20:58:52.256521+0000","last_fullsized":"2024-09-10T20:58:52.256521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:36.307037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258303+0000","last_change":"2024-09-10T20:58:47.246700+0000","last_active":"2024-09-10T20:58:52.258303+0000","last_peered":"2024-09-10T20:58:52.258303+0000","last_clean":"2024-09-10T20:58:52.258303+0000","last_became_active":"2024-09-10T20:58:47.246456+0000","last_became_peered":"2024-09-10T20:58:47.246456+0000","last_unstale":"2024-09-10T20:58:52.258303+0000","last_undegraded":"2024-09-10T20:58:52.258303+0000","last_fullsized":"2024-09-10T20:58:52.258303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:08.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316714+0000","last_change":"2024-09-10T20:58:46.224904+0000","last_active":"2024-09-10T20:58:53.316714+0000","last_peered":"2024-09-10T20:58:53.316714+0000","last_clean":"2024-09-10T20:58:53.316714+0000","last_became_active":"2024-09-10T20:58:46.224619+0000","last_became_peered":"2024-09-10T20:58:46.224619+0000","last_unstale":"2024-09-10T20:58:53.316714+0000","last_undegraded":"2024-09-10T20:58:53.316714+0000","last_fullsized":"2024-09-10T20:58:53.316714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:34.223018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256285+0000","last_change":"2024-09-10T20:58:51.274096+0000","last_active":"2024-09-10T20:58:52.256285+0000","last_peered":"2024-09-10T20:58:52.256285+0000","last_clean":"2024-09-10T20:58:52.256285+0000","last_became_active":"2024-09-10T20:58:51.273636+0000","last_became_peered":"2024-09-10T20:58:51.273636+0000","last_unstale":"2024-09-10T20:58:52.256285+0000","last_undegraded":"2024-09-10T20:58:52.256285+0000","last_fullsized":"2024-09-10T20:58:52.256285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:05.160808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258649+0000","last_change":"2024-09-10T20:58:47.228381+0000","last_active":"2024-09-10T20:58:52.258649+0000","last_peered":"2024-09-10T20:58:52.258649+0000","last_clean":"2024-09-10T20:58:52.258649+0000","last_became_active":"2024-09-10T20:58:47.228080+0000","last_became_peered":"2024-09-10T20:58:47.228080+0000","last_unstale":"2024-09-10T20:58:52.258649+0000","last_undegraded":"2024-09-10T20:58:52.258649+0000","last_fullsized":"2024-09-10T20:58:52.258649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:39.726666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316453+0000","last_change":"2024-09-10T20:58:46.226903+0000","last_active":"2024-09-10T20:58:53.316453+0000","last_peered":"2024-09-10T20:58:53.316453+0000","last_clean":"2024-09-10T20:58:53.316453+0000","last_became_active":"2024-09-10T20:58:46.226649+0000","last_became_peered":"2024-09-10T20:58:46.226649+0000","last_unstale":"2024-09-10T20:58:53.316453+0000","last_undegraded":"2024-09-10T20:58:53.316453+0000","last_fullsized":"2024-09-10T20:58:53.316453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:14.748746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317126+0000","last_change":"2024-09-10T20:58:51.554471+0000","last_active":"2024-09-10T20:58:53.317126+0000","last_peered":"2024-09-10T20:58:53.317126+0000","last_clean":"2024-09-10T20:58:53.317126+0000","last_became_active":"2024-09-10T20:58:51.554080+0000","last_became_peered":"2024-09-10T20:58:51.554080+0000","last_unstale":"2024-09-10T20:58:53.317126+0000","last_undegraded":"2024-09-10T20:58:53.317126+0000","last_fullsized":"2024-09-10T20:58:53.317126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:24.547698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256775+0000","last_change":"2024-09-10T20:58:47.234236+0000","last_active":"2024-09-10T20:58:52.256775+0000","last_peered":"2024-09-10T20:58:52.256775+0000","last_clean":"2024-09-10T20:58:52.256775+0000","last_became_active":"2024-09-10T20:58:47.233924+0000","last_became_peered":"2024-09-10T20:58:47.233924+0000","last_unstale":"2024-09-10T20:58:52.256775+0000","last_undegraded":"2024-09-10T20:58:52.256775+0000","last_fullsized":"2024-09-10T20:58:52.256775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:38.992405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903782+0000","last_change":"2024-09-10T20:58:46.221112+0000","last_active":"2024-09-10T20:58:53.903782+0000","last_peered":"2024-09-10T20:58:53.903782+0000","last_clean":"2024-09-10T20:58:53.903782+0000","last_became_active":"2024-09-10T20:58:46.220837+0000","last_became_peered":"2024-09-10T20:58:46.220837+0000","last_unstale":"2024-09-10T20:58:53.903782+0000","last_undegraded":"2024-09-10T20:58:53.903782+0000","last_fullsized":"2024-09-10T20:58:53.903782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:27.129152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256427+0000","last_change":"2024-09-10T20:58:51.274965+0000","last_active":"2024-09-10T20:58:52.256427+0000","last_peered":"2024-09-10T20:58:52.256427+0000","last_clean":"2024-09-10T20:58:52.256427+0000","last_became_active":"2024-09-10T20:58:51.274557+0000","last_became_peered":"2024-09-10T20:58:51.274557+0000","last_unstale":"2024-09-10T20:58:52.256427+0000","last_undegraded":"2024-09-10T20:58:52.256427+0000","last_fullsized":"2024-09-10T20:58:52.256427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:16.787797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904100+0000","last_change":"2024-09-10T20:58:47.246893+0000","last_active":"2024-09-10T20:58:53.904100+0000","last_peered":"2024-09-10T20:58:53.904100+0000","last_clean":"2024-09-10T20:58:53.904100+0000","last_became_active":"2024-09-10T20:58:47.245226+0000","last_became_peered":"2024-09-10T20:58:47.245226+0000","last_unstale":"2024-09-10T20:58:53.904100+0000","last_undegraded":"2024-09-10T20:58:53.904100+0000","last_fullsized":"2024-09-10T20:58:53.904100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:58.155371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904645+0000","last_change":"2024-09-10T20:58:46.222772+0000","last_active":"2024-09-10T20:58:53.904645+0000","last_peered":"2024-09-10T20:58:53.904645+0000","last_clean":"2024-09-10T20:58:53.904645+0000","last_became_active":"2024-09-10T20:58:46.222461+0000","last_became_peered":"2024-09-10T20:58:46.222461+0000","last_unstale":"2024-09-10T20:58:53.904645+0000","last_undegraded":"2024-09-10T20:58:53.904645+0000","last_fullsized":"2024-09-10T20:58:53.904645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:28.108849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256282+0000","last_change":"2024-09-10T20:58:51.554529+0000","last_active":"2024-09-10T20:58:52.256282+0000","last_peered":"2024-09-10T20:58:52.256282+0000","last_clean":"2024-09-10T20:58:52.256282+0000","last_became_active":"2024-09-10T20:58:51.554104+0000","last_became_peered":"2024-09-10T20:58:51.554104+0000","last_unstale":"2024-09-10T20:58:52.256282+0000","last_undegraded":"2024-09-10T20:58:52.256282+0000","last_fullsized":"2024-09-10T20:58:52.256282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:18.603248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.259179+0000","last_change":"2024-09-10T20:58:47.249780+0000","last_active":"2024-09-10T20:58:52.259179+0000","last_peered":"2024-09-10T20:58:52.259179+0000","last_clean":"2024-09-10T20:58:52.259179+0000","last_became_active":"2024-09-10T20:58:47.249519+0000","last_became_peered":"2024-09-10T20:58:47.249519+0000","last_unstale":"2024-09-10T20:58:52.259179+0000","last_undegraded":"2024-09-10T20:58:52.259179+0000","last_fullsized":"2024-09-10T20:58:52.259179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:58.169931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316888+0000","last_change":"2024-09-10T20:58:46.225535+0000","last_active":"2024-09-10T20:58:53.316888+0000","last_peered":"2024-09-10T20:58:53.316888+0000","last_clean":"2024-09-10T20:58:53.316888+0000","last_became_active":"2024-09-10T20:58:46.225248+0000","last_became_peered":"2024-09-10T20:58:46.225248+0000","last_unstale":"2024-09-10T20:58:53.316888+0000","last_undegraded":"2024-09-10T20:58:53.316888+0000","last_fullsized":"2024-09-10T20:58:53.316888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:22.880233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258298+0000","last_change":"2024-09-10T20:58:51.559242+0000","last_active":"2024-09-10T20:58:52.258298+0000","last_peered":"2024-09-10T20:58:52.258298+0000","last_clean":"2024-09-10T20:58:52.258298+0000","last_became_active":"2024-09-10T20:58:51.558875+0000","last_became_peered":"2024-09-10T20:58:51.558875+0000","last_unstale":"2024-09-10T20:58:52.258298+0000","last_undegraded":"2024-09-10T20:58:52.258298+0000","last_fullsized":"2024-09-10T20:58:52.258298+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:18.246776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317045+0000","last_change":"2024-09-10T20:58:47.228376+0000","last_active":"2024-09-10T20:58:53.317045+0000","last_peered":"2024-09-10T20:58:53.317045+0000","last_clean":"2024-09-10T20:58:53.317045+0000","last_became_active":"2024-09-10T20:58:47.228086+0000","last_became_peered":"2024-09-10T20:58:47.228086+0000","last_unstale":"2024-09-10T20:58:53.317045+0000","last_undegraded":"2024-09-10T20:58:53.317045+0000","last_fullsized":"2024-09-10T20:58:53.317045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:07.109711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905238+0000","last_change":"2024-09-10T20:58:46.221847+0000","last_active":"2024-09-10T20:58:53.905238+0000","last_peered":"2024-09-10T20:58:53.905238+0000","last_clean":"2024-09-10T20:58:53.905238+0000","last_became_active":"2024-09-10T20:58:46.221571+0000","last_became_peered":"2024-09-10T20:58:46.221571+0000","last_unstale":"2024-09-10T20:58:53.905238+0000","last_undegraded":"2024-09-10T20:58:53.905238+0000","last_fullsized":"2024-09-10T20:58:53.905238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:44.242563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903819+0000","last_change":"2024-09-10T20:58:51.554547+0000","last_active":"2024-09-10T20:58:53.903819+0000","last_peered":"2024-09-10T20:58:53.903819+0000","last_clean":"2024-09-10T20:58:53.903819+0000","last_became_active":"2024-09-10T20:58:51.554197+0000","last_became_peered":"2024-09-10T20:58:51.554197+0000","last_unstale":"2024-09-10T20:58:53.903819+0000","last_undegraded":"2024-09-10T20:58:53.903819+0000","last_fullsized":"2024-09-10T20:58:53.903819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:25.499034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256428+0000","last_change":"2024-09-10T20:58:47.243797+0000","last_active":"2024-09-10T20:58:52.256428+0000","last_peered":"2024-09-10T20:58:52.256428+0000","last_clean":"2024-09-10T20:58:52.256428+0000","last_became_active":"2024-09-10T20:58:47.243539+0000","last_became_peered":"2024-09-10T20:58:47.243539+0000","last_unstale":"2024-09-10T20:58:52.256428+0000","last_undegraded":"2024-09-10T20:58:52.256428+0000","last_fullsized":"2024-09-10T20:58:52.256428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:30.226963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317072+0000","last_change":"2024-09-10T20:58:46.221373+0000","last_active":"2024-09-10T20:58:53.317072+0000","last_peered":"2024-09-10T20:58:53.317072+0000","last_clean":"2024-09-10T20:58:53.317072+0000","last_became_active":"2024-09-10T20:58:46.221104+0000","last_became_peered":"2024-09-10T20:58:46.221104+0000","last_unstale":"2024-09-10T20:58:53.317072+0000","last_undegraded":"2024-09-10T20:58:53.317072+0000","last_fullsized":"2024-09-10T20:58:53.317072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:29.110234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316552+0000","last_change":"2024-09-10T20:58:51.278666+0000","last_active":"2024-09-10T20:58:53.316552+0000","last_peered":"2024-09-10T20:58:53.316552+0000","last_clean":"2024-09-10T20:58:53.316552+0000","last_became_active":"2024-09-10T20:58:51.278033+0000","last_became_peered":"2024-09-10T20:58:51.278033+0000","last_unstale":"2024-09-10T20:58:53.316552+0000","last_undegraded":"2024-09-10T20:58:53.316552+0000","last_fullsized":"2024-09-10T20:58:53.316552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:59.252455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257022+0000","last_change":"2024-09-10T20:58:47.246884+0000","last_active":"2024-09-10T20:58:52.257022+0000","last_peered":"2024-09-10T20:58:52.257022+0000","last_clean":"2024-09-10T20:58:52.257022+0000","last_became_active":"2024-09-10T20:58:47.246617+0000","last_became_peered":"2024-09-10T20:58:47.246617+0000","last_unstale":"2024-09-10T20:58:52.257022+0000","last_undegraded":"2024-09-10T20:58:52.257022+0000","last_fullsized":"2024-09-10T20:58:52.257022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:32.318955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256070+0000","last_change":"2024-09-10T20:58:46.225503+0000","last_active":"2024-09-10T20:58:52.256070+0000","last_peered":"2024-09-10T20:58:52.256070+0000","last_clean":"2024-09-10T20:58:52.256070+0000","last_became_active":"2024-09-10T20:58:46.225088+0000","last_became_peered":"2024-09-10T20:58:46.225088+0000","last_unstale":"2024-09-10T20:58:52.256070+0000","last_undegraded":"2024-09-10T20:58:52.256070+0000","last_fullsized":"2024-09-10T20:58:52.256070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:36.394527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317025+0000","last_change":"2024-09-10T20:58:51.277749+0000","last_active":"2024-09-10T20:58:53.317025+0000","last_peered":"2024-09-10T20:58:53.317025+0000","last_clean":"2024-09-10T20:58:53.317025+0000","last_became_active":"2024-09-10T20:58:51.277142+0000","last_became_peered":"2024-09-10T20:58:51.277142+0000","last_unstale":"2024-09-10T20:58:53.317025+0000","last_undegraded":"2024-09-10T20:58:53.317025+0000","last_fullsized":"2024-09-10T20:58:53.317025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:42.740546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316380+0000","last_change":"2024-09-10T20:58:47.228660+0000","last_active":"2024-09-10T20:58:53.316380+0000","last_peered":"2024-09-10T20:58:53.316380+0000","last_clean":"2024-09-10T20:58:53.316380+0000","last_became_active":"2024-09-10T20:58:47.228218+0000","last_became_peered":"2024-09-10T20:58:47.228218+0000","last_unstale":"2024-09-10T20:58:53.316380+0000","last_undegraded":"2024-09-10T20:58:53.316380+0000","last_fullsized":"2024-09-10T20:58:53.316380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:51.280929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903422+0000","last_change":"2024-09-10T20:58:46.221228+0000","last_active":"2024-09-10T20:58:53.903422+0000","last_peered":"2024-09-10T20:58:53.903422+0000","last_clean":"2024-09-10T20:58:53.903422+0000","last_became_active":"2024-09-10T20:58:46.220898+0000","last_became_peered":"2024-09-10T20:58:46.220898+0000","last_unstale":"2024-09-10T20:58:53.903422+0000","last_undegraded":"2024-09-10T20:58:53.903422+0000","last_fullsized":"2024-09-10T20:58:53.903422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:18.355281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317144+0000","last_change":"2024-09-10T20:58:51.274949+0000","last_active":"2024-09-10T20:58:53.317144+0000","last_peered":"2024-09-10T20:58:53.317144+0000","last_clean":"2024-09-10T20:58:53.317144+0000","last_became_active":"2024-09-10T20:58:51.274579+0000","last_became_peered":"2024-09-10T20:58:51.274579+0000","last_unstale":"2024-09-10T20:58:53.317144+0000","last_undegraded":"2024-09-10T20:58:53.317144+0000","last_fullsized":"2024-09-10T20:58:53.317144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:50.176038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903242+0000","last_change":"2024-09-10T20:58:47.240914+0000","last_active":"2024-09-10T20:58:53.903242+0000","last_peered":"2024-09-10T20:58:53.903242+0000","last_clean":"2024-09-10T20:58:53.903242+0000","last_became_active":"2024-09-10T20:58:47.240250+0000","last_became_peered":"2024-09-10T20:58:47.240250+0000","last_unstale":"2024-09-10T20:58:53.903242+0000","last_undegraded":"2024-09-10T20:58:53.903242+0000","last_fullsized":"2024-09-10T20:58:53.903242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:46.184593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903948+0000","last_change":"2024-09-10T20:58:46.224644+0000","last_active":"2024-09-10T20:58:53.903948+0000","last_peered":"2024-09-10T20:58:53.903948+0000","last_clean":"2024-09-10T20:58:53.903948+0000","last_became_active":"2024-09-10T20:58:46.224329+0000","last_became_peered":"2024-09-10T20:58:46.224329+0000","last_unstale":"2024-09-10T20:58:53.903948+0000","last_undegraded":"2024-09-10T20:58:53.903948+0000","last_fullsized":"2024-09-10T20:58:53.903948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:01.590603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904042+0000","last_change":"2024-09-10T20:58:51.554468+0000","last_active":"2024-09-10T20:58:53.904042+0000","last_peered":"2024-09-10T20:58:53.904042+0000","last_clean":"2024-09-10T20:58:53.904042+0000","last_became_active":"2024-09-10T20:58:51.554092+0000","last_became_peered":"2024-09-10T20:58:51.554092+0000","last_unstale":"2024-09-10T20:58:53.904042+0000","last_undegraded":"2024-09-10T20:58:53.904042+0000","last_fullsized":"2024-09-10T20:58:53.904042+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:48.958938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258417+0000","last_change":"2024-09-10T20:58:47.242758+0000","last_active":"2024-09-10T20:58:52.258417+0000","last_peered":"2024-09-10T20:58:52.258417+0000","last_clean":"2024-09-10T20:58:52.258417+0000","last_became_active":"2024-09-10T20:58:47.242439+0000","last_became_peered":"2024-09-10T20:58:47.242439+0000","last_unstale":"2024-09-10T20:58:52.258417+0000","last_undegraded":"2024-09-10T20:58:52.258417+0000","last_fullsized":"2024-09-10T20:58:52.258417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:45.212911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255997+0000","last_change":"2024-09-10T20:58:46.222264+0000","last_active":"2024-09-10T20:58:52.255997+0000","last_peered":"2024-09-10T20:58:52.255997+0000","last_clean":"2024-09-10T20:58:52.255997+0000","last_became_active":"2024-09-10T20:58:46.221414+0000","last_became_peered":"2024-09-10T20:58:46.221414+0000","last_unstale":"2024-09-10T20:58:52.255997+0000","last_undegraded":"2024-09-10T20:58:52.255997+0000","last_fullsized":"2024-09-10T20:58:52.255997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:22.019957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":119,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904053+0000","last_change":"2024-09-10T20:58:36.895993+0000","last_active":"2024-09-10T20:58:53.904053+0000","last_peered":"2024-09-10T20:58:53.904053+0000","last_clean":"2024-09-10T20:58:53.904053+0000","last_became_active":"2024-09-10T20:58:36.894988+0000","last_became_peered":"2024-09-10T20:58:36.894988+0000","last_unstale":"2024-09-10T20:58:53.904053+0000","last_undegraded":"2024-09-10T20:58:53.904053+0000","last_fullsized":"2024-09-10T20:58:53.904053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_clean_scrub_stamp":"2024-09-10T20:58:35.588702+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.082528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904143+0000","last_change":"2024-09-10T20:58:51.556599+0000","last_active":"2024-09-10T20:58:53.904143+0000","last_peered":"2024-09-10T20:58:53.904143+0000","last_clean":"2024-09-10T20:58:53.904143+0000","last_became_active":"2024-09-10T20:58:51.556210+0000","last_became_peered":"2024-09-10T20:58:51.556210+0000","last_unstale":"2024-09-10T20:58:53.904143+0000","last_undegraded":"2024-09-10T20:58:53.904143+0000","last_fullsized":"2024-09-10T20:58:53.904143+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:36.307037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904986+0000","last_change":"2024-09-10T20:58:47.245638+0000","last_active":"2024-09-10T20:58:53.904986+0000","last_peered":"2024-09-10T20:58:53.904986+0000","last_clean":"2024-09-10T20:58:53.904986+0000","last_became_active":"2024-09-10T20:58:47.244988+0000","last_became_peered":"2024-09-10T20:58:47.244988+0000","last_unstale":"2024-09-10T20:58:53.904986+0000","last_undegraded":"2024-09-10T20:58:53.904986+0000","last_fullsized":"2024-09-10T20:58:53.904986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:04.445410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903834+0000","last_change":"2024-09-10T20:58:46.228090+0000","last_active":"2024-09-10T20:58:53.903834+0000","last_peered":"2024-09-10T20:58:53.903834+0000","last_clean":"2024-09-10T20:58:53.903834+0000","last_became_active":"2024-09-10T20:58:46.227789+0000","last_became_peered":"2024-09-10T20:58:46.227789+0000","last_unstale":"2024-09-10T20:58:53.903834+0000","last_undegraded":"2024-09-10T20:58:53.903834+0000","last_fullsized":"2024-09-10T20:58:53.903834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:37.064288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904078+0000","last_change":"2024-09-10T20:58:51.554553+0000","last_active":"2024-09-10T20:58:53.904078+0000","last_peered":"2024-09-10T20:58:53.904078+0000","last_clean":"2024-09-10T20:58:53.904078+0000","last_became_active":"2024-09-10T20:58:51.554131+0000","last_became_peered":"2024-09-10T20:58:51.554131+0000","last_unstale":"2024-09-10T20:58:53.904078+0000","last_undegraded":"2024-09-10T20:58:53.904078+0000","last_fullsized":"2024-09-10T20:58:53.904078+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:18.603248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255727+0000","last_change":"2024-09-10T20:58:47.245802+0000","last_active":"2024-09-10T20:58:52.255727+0000","last_peered":"2024-09-10T20:58:52.255727+0000","last_clean":"2024-09-10T20:58:52.255727+0000","last_became_active":"2024-09-10T20:58:47.245543+0000","last_became_peered":"2024-09-10T20:58:47.245543+0000","last_unstale":"2024-09-10T20:58:52.255727+0000","last_undegraded":"2024-09-10T20:58:52.255727+0000","last_fullsized":"2024-09-10T20:58:52.255727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:17.787548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316844+0000","last_change":"2024-09-10T20:58:46.226841+0000","last_active":"2024-09-10T20:58:53.316844+0000","last_peered":"2024-09-10T20:58:53.316844+0000","last_clean":"2024-09-10T20:58:53.316844+0000","last_became_active":"2024-09-10T20:58:46.226610+0000","last_became_peered":"2024-09-10T20:58:46.226610+0000","last_unstale":"2024-09-10T20:58:53.316844+0000","last_undegraded":"2024-09-10T20:58:53.316844+0000","last_fullsized":"2024-09-10T20:58:53.316844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:41.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904125+0000","last_change":"2024-09-10T20:58:51.274667+0000","last_active":"2024-09-10T20:58:53.904125+0000","last_peered":"2024-09-10T20:58:53.904125+0000","last_clean":"2024-09-10T20:58:53.904125+0000","last_became_active":"2024-09-10T20:58:51.274284+0000","last_became_peered":"2024-09-10T20:58:51.274284+0000","last_unstale":"2024-09-10T20:58:53.904125+0000","last_undegraded":"2024-09-10T20:58:53.904125+0000","last_fullsized":"2024-09-10T20:58:53.904125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:36.498874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905137+0000","last_change":"2024-09-10T20:58:47.245680+0000","last_active":"2024-09-10T20:58:53.905137+0000","last_peered":"2024-09-10T20:58:53.905137+0000","last_clean":"2024-09-10T20:58:53.905137+0000","last_became_active":"2024-09-10T20:58:47.245058+0000","last_became_peered":"2024-09-10T20:58:47.245058+0000","last_unstale":"2024-09-10T20:58:53.905137+0000","last_undegraded":"2024-09-10T20:58:53.905137+0000","last_fullsized":"2024-09-10T20:58:53.905137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:36.576513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903742+0000","last_change":"2024-09-10T20:58:46.222762+0000","last_active":"2024-09-10T20:58:53.903742+0000","last_peered":"2024-09-10T20:58:53.903742+0000","last_clean":"2024-09-10T20:58:53.903742+0000","last_became_active":"2024-09-10T20:58:46.222478+0000","last_became_peered":"2024-09-10T20:58:46.222478+0000","last_unstale":"2024-09-10T20:58:53.903742+0000","last_undegraded":"2024-09-10T20:58:53.903742+0000","last_fullsized":"2024-09-10T20:58:53.903742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:20.646322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256423+0000","last_change":"2024-09-10T20:58:51.554839+0000","last_active":"2024-09-10T20:58:52.256423+0000","last_peered":"2024-09-10T20:58:52.256423+0000","last_clean":"2024-09-10T20:58:52.256423+0000","last_became_active":"2024-09-10T20:58:51.554371+0000","last_became_peered":"2024-09-10T20:58:51.554371+0000","last_unstale":"2024-09-10T20:58:52.256423+0000","last_undegraded":"2024-09-10T20:58:52.256423+0000","last_fullsized":"2024-09-10T20:58:52.256423+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:16.242767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905199+0000","last_change":"2024-09-10T20:58:47.239517+0000","last_active":"2024-09-10T20:58:53.905199+0000","last_peered":"2024-09-10T20:58:53.905199+0000","last_clean":"2024-09-10T20:58:53.905199+0000","last_became_active":"2024-09-10T20:58:47.239275+0000","last_became_peered":"2024-09-10T20:58:47.239275+0000","last_unstale":"2024-09-10T20:58:53.905199+0000","last_undegraded":"2024-09-10T20:58:53.905199+0000","last_fullsized":"2024-09-10T20:58:53.905199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:27.143705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256315+0000","last_change":"2024-09-10T20:58:46.226170+0000","last_active":"2024-09-10T20:58:52.256315+0000","last_peered":"2024-09-10T20:58:52.256315+0000","last_clean":"2024-09-10T20:58:52.256315+0000","last_became_active":"2024-09-10T20:58:46.225888+0000","last_became_peered":"2024-09-10T20:58:46.225888+0000","last_unstale":"2024-09-10T20:58:52.256315+0000","last_undegraded":"2024-09-10T20:58:52.256315+0000","last_fullsized":"2024-09-10T20:58:52.256315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:27.108377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255461+0000","last_change":"2024-09-10T20:58:51.276340+0000","last_active":"2024-09-10T20:58:52.255461+0000","last_peered":"2024-09-10T20:58:52.255461+0000","last_clean":"2024-09-10T20:58:52.255461+0000","last_became_active":"2024-09-10T20:58:51.275903+0000","last_became_peered":"2024-09-10T20:58:51.275903+0000","last_unstale":"2024-09-10T20:58:52.255461+0000","last_undegraded":"2024-09-10T20:58:52.255461+0000","last_fullsized":"2024-09-10T20:58:52.255461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:04.361773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904252+0000","last_change":"2024-09-10T20:58:47.237276+0000","last_active":"2024-09-10T20:58:53.904252+0000","last_peered":"2024-09-10T20:58:53.904252+0000","last_clean":"2024-09-10T20:58:53.904252+0000","last_became_active":"2024-09-10T20:58:47.235334+0000","last_became_peered":"2024-09-10T20:58:47.235334+0000","last_unstale":"2024-09-10T20:58:53.904252+0000","last_undegraded":"2024-09-10T20:58:53.904252+0000","last_fullsized":"2024-09-10T20:58:53.904252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:58.169931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903190+0000","last_change":"2024-09-10T20:58:46.224350+0000","last_active":"2024-09-10T20:58:53.903190+0000","last_peered":"2024-09-10T20:58:53.903190+0000","last_clean":"2024-09-10T20:58:53.903190+0000","last_became_active":"2024-09-10T20:58:46.223931+0000","last_became_peered":"2024-09-10T20:58:46.223931+0000","last_unstale":"2024-09-10T20:58:53.903190+0000","last_undegraded":"2024-09-10T20:58:53.903190+0000","last_fullsized":"2024-09-10T20:58:53.903190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:22.019957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903918+0000","last_change":"2024-09-10T20:58:51.270969+0000","last_active":"2024-09-10T20:58:53.903918+0000","last_peered":"2024-09-10T20:58:53.903918+0000","last_clean":"2024-09-10T20:58:53.903918+0000","last_became_active":"2024-09-10T20:58:51.270229+0000","last_became_peered":"2024-09-10T20:58:51.270229+0000","last_unstale":"2024-09-10T20:58:53.903918+0000","last_undegraded":"2024-09-10T20:58:53.903918+0000","last_fullsized":"2024-09-10T20:58:53.903918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:35.803700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903940+0000","last_change":"2024-09-10T20:58:47.247096+0000","last_active":"2024-09-10T20:58:53.903940+0000","last_peered":"2024-09-10T20:58:53.903940+0000","last_clean":"2024-09-10T20:58:53.903940+0000","last_became_active":"2024-09-10T20:58:47.246797+0000","last_became_peered":"2024-09-10T20:58:47.246797+0000","last_unstale":"2024-09-10T20:58:53.903940+0000","last_undegraded":"2024-09-10T20:58:53.903940+0000","last_fullsized":"2024-09-10T20:58:53.903940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:18.950297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904318+0000","last_change":"2024-09-10T20:58:46.226890+0000","last_active":"2024-09-10T20:58:53.904318+0000","last_peered":"2024-09-10T20:58:53.904318+0000","last_clean":"2024-09-10T20:58:53.904318+0000","last_became_active":"2024-09-10T20:58:46.226615+0000","last_became_peered":"2024-09-10T20:58:46.226615+0000","last_unstale":"2024-09-10T20:58:53.904318+0000","last_undegraded":"2024-09-10T20:58:53.904318+0000","last_fullsized":"2024-09-10T20:58:53.904318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:08.671158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.259085+0000","last_change":"2024-09-10T20:58:51.556517+0000","last_active":"2024-09-10T20:58:52.259085+0000","last_peered":"2024-09-10T20:58:52.259085+0000","last_clean":"2024-09-10T20:58:52.259085+0000","last_became_active":"2024-09-10T20:58:51.556152+0000","last_became_peered":"2024-09-10T20:58:51.556152+0000","last_unstale":"2024-09-10T20:58:52.259085+0000","last_undegraded":"2024-09-10T20:58:52.259085+0000","last_fullsized":"2024-09-10T20:58:52.259085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:46.913894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904720+0000","last_change":"2024-09-10T20:58:47.245297+0000","last_active":"2024-09-10T20:58:53.904720+0000","last_peered":"2024-09-10T20:58:53.904720+0000","last_clean":"2024-09-10T20:58:53.904720+0000","last_became_active":"2024-09-10T20:58:47.244947+0000","last_became_peered":"2024-09-10T20:58:47.244947+0000","last_unstale":"2024-09-10T20:58:53.904720+0000","last_undegraded":"2024-09-10T20:58:53.904720+0000","last_fullsized":"2024-09-10T20:58:53.904720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:25.115061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255795+0000","last_change":"2024-09-10T20:58:46.229169+0000","last_active":"2024-09-10T20:58:52.255795+0000","last_peered":"2024-09-10T20:58:52.255795+0000","last_clean":"2024-09-10T20:58:52.255795+0000","last_became_active":"2024-09-10T20:58:46.228622+0000","last_became_peered":"2024-09-10T20:58:46.228622+0000","last_unstale":"2024-09-10T20:58:52.255795+0000","last_undegraded":"2024-09-10T20:58:52.255795+0000","last_fullsized":"2024-09-10T20:58:52.255795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:35.810342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258536+0000","last_change":"2024-09-10T20:58:51.554436+0000","last_active":"2024-09-10T20:58:52.258536+0000","last_peered":"2024-09-10T20:58:52.258536+0000","last_clean":"2024-09-10T20:58:52.258536+0000","last_became_active":"2024-09-10T20:58:51.554038+0000","last_became_peered":"2024-09-10T20:58:51.554038+0000","last_unstale":"2024-09-10T20:58:52.258536+0000","last_undegraded":"2024-09-10T20:58:52.258536+0000","last_fullsized":"2024-09-10T20:58:52.258536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:33.352140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903833+0000","last_change":"2024-09-10T20:58:47.247094+0000","last_active":"2024-09-10T20:58:53.903833+0000","last_peered":"2024-09-10T20:58:53.903833+0000","last_clean":"2024-09-10T20:58:53.903833+0000","last_became_active":"2024-09-10T20:58:47.246095+0000","last_became_peered":"2024-09-10T20:58:47.246095+0000","last_unstale":"2024-09-10T20:58:53.903833+0000","last_undegraded":"2024-09-10T20:58:53.903833+0000","last_fullsized":"2024-09-10T20:58:53.903833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:39.726666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904184+0000","last_change":"2024-09-10T20:58:46.226623+0000","last_active":"2024-09-10T20:58:53.904184+0000","last_peered":"2024-09-10T20:58:53.904184+0000","last_clean":"2024-09-10T20:58:53.904184+0000","last_became_active":"2024-09-10T20:58:46.226326+0000","last_became_peered":"2024-09-10T20:58:46.226326+0000","last_unstale":"2024-09-10T20:58:53.904184+0000","last_undegraded":"2024-09-10T20:58:53.904184+0000","last_fullsized":"2024-09-10T20:58:53.904184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:02:21.934804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256251+0000","last_change":"2024-09-10T20:58:51.556678+0000","last_active":"2024-09-10T20:58:52.256251+0000","last_peered":"2024-09-10T20:58:52.256251+0000","last_clean":"2024-09-10T20:58:52.256251+0000","last_became_active":"2024-09-10T20:58:51.556271+0000","last_became_peered":"2024-09-10T20:58:51.556271+0000","last_unstale":"2024-09-10T20:58:52.256251+0000","last_undegraded":"2024-09-10T20:58:52.256251+0000","last_fullsized":"2024-09-10T20:58:52.256251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:02.438361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904916+0000","last_change":"2024-09-10T20:58:47.243793+0000","last_active":"2024-09-10T20:58:53.904916+0000","last_peered":"2024-09-10T20:58:53.904916+0000","last_clean":"2024-09-10T20:58:53.904916+0000","last_became_active":"2024-09-10T20:58:47.243537+0000","last_became_peered":"2024-09-10T20:58:47.243537+0000","last_unstale":"2024-09-10T20:58:53.904916+0000","last_undegraded":"2024-09-10T20:58:53.904916+0000","last_fullsized":"2024-09-10T20:58:53.904916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:04.464193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:56.612638+0000","last_change":"2024-09-10T20:58:46.225562+0000","last_active":"2024-09-10T20:58:56.612638+0000","last_peered":"2024-09-10T20:58:56.612638+0000","last_clean":"2024-09-10T20:58:56.612638+0000","last_became_active":"2024-09-10T20:58:46.225260+0000","last_became_peered":"2024-09-10T20:58:46.225260+0000","last_unstale":"2024-09-10T20:58:56.612638+0000","last_undegraded":"2024-09-10T20:58:56.612638+0000","last_fullsized":"2024-09-10T20:58:56.612638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:31.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316695+0000","last_change":"2024-09-10T20:58:51.278993+0000","last_active":"2024-09-10T20:58:53.316695+0000","last_peered":"2024-09-10T20:58:53.316695+0000","last_clean":"2024-09-10T20:58:53.316695+0000","last_became_active":"2024-09-10T20:58:51.278069+0000","last_became_peered":"2024-09-10T20:58:51.278069+0000","last_unstale":"2024-09-10T20:58:53.316695+0000","last_undegraded":"2024-09-10T20:58:53.316695+0000","last_fullsized":"2024-09-10T20:58:53.316695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:21.828620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257045+0000","last_change":"2024-09-10T20:58:47.235717+0000","last_active":"2024-09-10T20:58:52.257045+0000","last_peered":"2024-09-10T20:58:52.257045+0000","last_clean":"2024-09-10T20:58:52.257045+0000","last_became_active":"2024-09-10T20:58:47.235451+0000","last_became_peered":"2024-09-10T20:58:47.235451+0000","last_unstale":"2024-09-10T20:58:52.257045+0000","last_undegraded":"2024-09-10T20:58:52.257045+0000","last_fullsized":"2024-09-10T20:58:52.257045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:28.134150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316771+0000","last_change":"2024-09-10T20:58:46.226035+0000","last_active":"2024-09-10T20:58:53.316771+0000","last_peered":"2024-09-10T20:58:53.316771+0000","last_clean":"2024-09-10T20:58:53.316771+0000","last_became_active":"2024-09-10T20:58:46.225778+0000","last_became_peered":"2024-09-10T20:58:46.225778+0000","last_unstale":"2024-09-10T20:58:53.316771+0000","last_undegraded":"2024-09-10T20:58:53.316771+0000","last_fullsized":"2024-09-10T20:58:53.316771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:57.175951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256534+0000","last_change":"2024-09-10T20:58:51.274974+0000","last_active":"2024-09-10T20:58:52.256534+0000","last_peered":"2024-09-10T20:58:52.256534+0000","last_clean":"2024-09-10T20:58:52.256534+0000","last_became_active":"2024-09-10T20:58:51.274597+0000","last_became_peered":"2024-09-10T20:58:51.274597+0000","last_unstale":"2024-09-10T20:58:52.256534+0000","last_undegraded":"2024-09-10T20:58:52.256534+0000","last_fullsized":"2024-09-10T20:58:52.256534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:31.184777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904058+0000","last_change":"2024-09-10T20:58:47.245724+0000","last_active":"2024-09-10T20:58:53.904058+0000","last_peered":"2024-09-10T20:58:53.904058+0000","last_clean":"2024-09-10T20:58:53.904058+0000","last_became_active":"2024-09-10T20:58:47.245252+0000","last_became_peered":"2024-09-10T20:58:47.245252+0000","last_unstale":"2024-09-10T20:58:53.904058+0000","last_undegraded":"2024-09-10T20:58:53.904058+0000","last_fullsized":"2024-09-10T20:58:53.904058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:29.836234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255635+0000","last_change":"2024-09-10T20:58:46.229938+0000","last_active":"2024-09-10T20:58:52.255635+0000","last_peered":"2024-09-10T20:58:52.255635+0000","last_clean":"2024-09-10T20:58:52.255635+0000","last_became_active":"2024-09-10T20:58:46.229121+0000","last_became_peered":"2024-09-10T20:58:46.229121+0000","last_unstale":"2024-09-10T20:58:52.255635+0000","last_undegraded":"2024-09-10T20:58:52.255635+0000","last_fullsized":"2024-09-10T20:58:52.255635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:18.355281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258591+0000","last_change":"2024-09-10T20:58:51.280436+0000","last_active":"2024-09-10T20:58:52.258591+0000","last_peered":"2024-09-10T20:58:52.258591+0000","last_clean":"2024-09-10T20:58:52.258591+0000","last_became_active":"2024-09-10T20:58:51.279981+0000","last_became_peered":"2024-09-10T20:58:51.279981+0000","last_unstale":"2024-09-10T20:58:52.258591+0000","last_undegraded":"2024-09-10T20:58:52.258591+0000","last_fullsized":"2024-09-10T20:58:52.258591+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:01.843001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255541+0000","last_change":"2024-09-10T20:58:47.235938+0000","last_active":"2024-09-10T20:58:52.255541+0000","last_peered":"2024-09-10T20:58:52.255541+0000","last_clean":"2024-09-10T20:58:52.255541+0000","last_became_active":"2024-09-10T20:58:47.235628+0000","last_became_peered":"2024-09-10T20:58:47.235628+0000","last_unstale":"2024-09-10T20:58:52.255541+0000","last_undegraded":"2024-09-10T20:58:52.255541+0000","last_fullsized":"2024-09-10T20:58:52.255541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:13.462491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904541+0000","last_change":"2024-09-10T20:58:46.230618+0000","last_active":"2024-09-10T20:58:53.904541+0000","last_peered":"2024-09-10T20:58:53.904541+0000","last_clean":"2024-09-10T20:58:53.904541+0000","last_became_active":"2024-09-10T20:58:46.230289+0000","last_became_peered":"2024-09-10T20:58:46.230289+0000","last_unstale":"2024-09-10T20:58:53.904541+0000","last_undegraded":"2024-09-10T20:58:53.904541+0000","last_fullsized":"2024-09-10T20:58:53.904541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:51.217167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258193+0000","last_change":"2024-09-10T20:58:51.557495+0000","last_active":"2024-09-10T20:58:52.258193+0000","last_peered":"2024-09-10T20:58:52.258193+0000","last_clean":"2024-09-10T20:58:52.258193+0000","last_became_active":"2024-09-10T20:58:51.556492+0000","last_became_peered":"2024-09-10T20:58:51.556492+0000","last_unstale":"2024-09-10T20:58:52.258193+0000","last_undegraded":"2024-09-10T20:58:52.258193+0000","last_fullsized":"2024-09-10T20:58:52.258193+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:48.716121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903643+0000","last_change":"2024-09-10T20:58:51.555588+0000","last_active":"2024-09-10T20:58:53.903643+0000","last_peered":"2024-09-10T20:58:53.903643+0000","last_clean":"2024-09-10T20:58:53.903643+0000","last_became_active":"2024-09-10T20:58:51.555232+0000","last_became_peered":"2024-09-10T20:58:51.555232+0000","last_unstale":"2024-09-10T20:58:53.903643+0000","last_undegraded":"2024-09-10T20:58:53.903643+0000","last_fullsized":"2024-09-10T20:58:53.903643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.486482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316315+0000","last_change":"2024-09-10T20:58:47.239189+0000","last_active":"2024-09-10T20:58:53.316315+0000","last_peered":"2024-09-10T20:58:53.316315+0000","last_clean":"2024-09-10T20:58:53.316315+0000","last_became_active":"2024-09-10T20:58:47.238926+0000","last_became_peered":"2024-09-10T20:58:47.238926+0000","last_unstale":"2024-09-10T20:58:53.316315+0000","last_undegraded":"2024-09-10T20:58:53.316315+0000","last_fullsized":"2024-09-10T20:58:53.316315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:02.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255350+0000","last_change":"2024-09-10T20:58:46.229369+0000","last_active":"2024-09-10T20:58:52.255350+0000","last_peered":"2024-09-10T20:58:52.255350+0000","last_clean":"2024-09-10T20:58:52.255350+0000","last_became_active":"2024-09-10T20:58:46.229094+0000","last_became_peered":"2024-09-10T20:58:46.229094+0000","last_unstale":"2024-09-10T20:58:52.255350+0000","last_undegraded":"2024-09-10T20:58:52.255350+0000","last_fullsized":"2024-09-10T20:58:52.255350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:20.646322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904390+0000","last_change":"2024-09-10T20:58:51.280161+0000","last_active":"2024-09-10T20:58:53.904390+0000","last_peered":"2024-09-10T20:58:53.904390+0000","last_clean":"2024-09-10T20:58:53.904390+0000","last_became_active":"2024-09-10T20:58:51.279714+0000","last_became_peered":"2024-09-10T20:58:51.279714+0000","last_unstale":"2024-09-10T20:58:53.904390+0000","last_undegraded":"2024-09-10T20:58:53.904390+0000","last_fullsized":"2024-09-10T20:58:53.904390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.486482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258577+0000","last_change":"2024-09-10T20:58:47.248033+0000","last_active":"2024-09-10T20:58:52.258577+0000","last_peered":"2024-09-10T20:58:52.258577+0000","last_clean":"2024-09-10T20:58:52.258577+0000","last_became_active":"2024-09-10T20:58:47.247761+0000","last_became_peered":"2024-09-10T20:58:47.247761+0000","last_unstale":"2024-09-10T20:58:52.258577+0000","last_undegraded":"2024-09-10T20:58:52.258577+0000","last_fullsized":"2024-09-10T20:58:52.258577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:04.498018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258357+0000","last_change":"2024-09-10T20:58:46.223392+0000","last_active":"2024-09-10T20:58:52.258357+0000","last_peered":"2024-09-10T20:58:52.258357+0000","last_clean":"2024-09-10T20:58:52.258357+0000","last_became_active":"2024-09-10T20:58:46.223123+0000","last_became_peered":"2024-09-10T20:58:46.223123+0000","last_unstale":"2024-09-10T20:58:52.258357+0000","last_undegraded":"2024-09-10T20:58:52.258357+0000","last_fullsized":"2024-09-10T20:58:52.258357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:16.570128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904433+0000","last_change":"2024-09-10T20:58:46.227651+0000","last_active":"2024-09-10T20:58:53.904433+0000","last_peered":"2024-09-10T20:58:53.904433+0000","last_clean":"2024-09-10T20:58:53.904433+0000","last_became_active":"2024-09-10T20:58:46.226418+0000","last_became_peered":"2024-09-10T20:58:46.226418+0000","last_unstale":"2024-09-10T20:58:53.904433+0000","last_undegraded":"2024-09-10T20:58:53.904433+0000","last_fullsized":"2024-09-10T20:58:53.904433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:02.951532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904158+0000","last_change":"2024-09-10T20:58:47.245394+0000","last_active":"2024-09-10T20:58:53.904158+0000","last_peered":"2024-09-10T20:58:53.904158+0000","last_clean":"2024-09-10T20:58:53.904158+0000","last_became_active":"2024-09-10T20:58:47.243973+0000","last_became_peered":"2024-09-10T20:58:47.243973+0000","last_unstale":"2024-09-10T20:58:53.904158+0000","last_undegraded":"2024-09-10T20:58:53.904158+0000","last_fullsized":"2024-09-10T20:58:53.904158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:54.993134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903643+0000","last_change":"2024-09-10T20:58:51.274539+0000","last_active":"2024-09-10T20:58:53.903643+0000","last_peered":"2024-09-10T20:58:53.903643+0000","last_clean":"2024-09-10T20:58:53.903643+0000","last_became_active":"2024-09-10T20:58:51.274124+0000","last_became_peered":"2024-09-10T20:58:51.274124+0000","last_unstale":"2024-09-10T20:58:53.903643+0000","last_undegraded":"2024-09-10T20:58:53.903643+0000","last_fullsized":"2024-09-10T20:58:53.903643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:20.469415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258431+0000","last_change":"2024-09-10T20:58:46.223353+0000","last_active":"2024-09-10T20:58:52.258431+0000","last_peered":"2024-09-10T20:58:52.258431+0000","last_clean":"2024-09-10T20:58:52.258431+0000","last_became_active":"2024-09-10T20:58:46.223059+0000","last_became_peered":"2024-09-10T20:58:46.223059+0000","last_unstale":"2024-09-10T20:58:52.258431+0000","last_undegraded":"2024-09-10T20:58:52.258431+0000","last_fullsized":"2024-09-10T20:58:52.258431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:02:21.934804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257112+0000","last_change":"2024-09-10T20:58:47.233445+0000","last_active":"2024-09-10T20:58:52.257112+0000","last_peered":"2024-09-10T20:58:52.257112+0000","last_clean":"2024-09-10T20:58:52.257112+0000","last_became_active":"2024-09-10T20:58:47.233157+0000","last_became_peered":"2024-09-10T20:58:47.233157+0000","last_unstale":"2024-09-10T20:58:52.257112+0000","last_undegraded":"2024-09-10T20:58:52.257112+0000","last_fullsized":"2024-09-10T20:58:52.257112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:52.222165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258673+0000","last_change":"2024-09-10T20:58:51.281498+0000","last_active":"2024-09-10T20:58:52.258673+0000","last_peered":"2024-09-10T20:58:52.258673+0000","last_clean":"2024-09-10T20:58:52.258673+0000","last_became_active":"2024-09-10T20:58:51.281027+0000","last_became_peered":"2024-09-10T20:58:51.281027+0000","last_unstale":"2024-09-10T20:58:52.258673+0000","last_undegraded":"2024-09-10T20:58:52.258673+0000","last_fullsized":"2024-09-10T20:58:52.258673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:35.279483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903782+0000","last_change":"2024-09-10T20:58:47.247788+0000","last_active":"2024-09-10T20:58:53.903782+0000","last_peered":"2024-09-10T20:58:53.903782+0000","last_clean":"2024-09-10T20:58:53.903782+0000","last_became_active":"2024-09-10T20:58:47.247513+0000","last_became_peered":"2024-09-10T20:58:47.247513+0000","last_unstale":"2024-09-10T20:58:53.903782+0000","last_undegraded":"2024-09-10T20:58:53.903782+0000","last_fullsized":"2024-09-10T20:58:53.903782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:59.707834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903324+0000","last_change":"2024-09-10T20:58:47.232038+0000","last_active":"2024-09-10T20:58:53.903324+0000","last_peered":"2024-09-10T20:58:53.903324+0000","last_clean":"2024-09-10T20:58:53.903324+0000","last_became_active":"2024-09-10T20:58:47.231590+0000","last_became_peered":"2024-09-10T20:58:47.231590+0000","last_unstale":"2024-09-10T20:58:53.903324+0000","last_undegraded":"2024-09-10T20:58:53.903324+0000","last_fullsized":"2024-09-10T20:58:53.903324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:26.466044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255659+0000","last_change":"2024-09-10T20:58:47.237625+0000","last_active":"2024-09-10T20:58:52.255659+0000","last_peered":"2024-09-10T20:58:52.255659+0000","last_clean":"2024-09-10T20:58:52.255659+0000","last_became_active":"2024-09-10T20:58:47.237247+0000","last_became_peered":"2024-09-10T20:58:47.237247+0000","last_unstale":"2024-09-10T20:58:52.255659+0000","last_undegraded":"2024-09-10T20:58:52.255659+0000","last_fullsized":"2024-09-10T20:58:52.255659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:21.642999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904061+0000","last_change":"2024-09-10T20:58:47.247495+0000","last_active":"2024-09-10T20:58:53.904061+0000","last_peered":"2024-09-10T20:58:53.904061+0000","last_clean":"2024-09-10T20:58:53.904061+0000","last_became_active":"2024-09-10T20:58:47.247049+0000","last_became_peered":"2024-09-10T20:58:47.247049+0000","last_unstale":"2024-09-10T20:58:53.904061+0000","last_undegraded":"2024-09-10T20:58:53.904061+0000","last_fullsized":"2024-09-10T20:58:53.904061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:18.950297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258717+0000","last_change":"2024-09-10T20:58:47.243173+0000","last_active":"2024-09-10T20:58:52.258717+0000","last_peered":"2024-09-10T20:58:52.258717+0000","last_clean":"2024-09-10T20:58:52.258717+0000","last_became_active":"2024-09-10T20:58:47.242521+0000","last_became_peered":"2024-09-10T20:58:47.242521+0000","last_unstale":"2024-09-10T20:58:52.258717+0000","last_undegraded":"2024-09-10T20:58:52.258717+0000","last_fullsized":"2024-09-10T20:58:52.258717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:54.993134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316600+0000","last_change":"2024-09-10T20:58:47.238462+0000","last_active":"2024-09-10T20:58:53.316600+0000","last_peered":"2024-09-10T20:58:53.316600+0000","last_clean":"2024-09-10T20:58:53.316600+0000","last_became_active":"2024-09-10T20:58:47.238166+0000","last_became_peered":"2024-09-10T20:58:47.238166+0000","last_unstale":"2024-09-10T20:58:53.316600+0000","last_undegraded":"2024-09-10T20:58:53.316600+0000","last_fullsized":"2024-09-10T20:58:53.316600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:51.719957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904758+0000","last_change":"2024-09-10T20:58:47.245403+0000","last_active":"2024-09-10T20:58:53.904758+0000","last_peered":"2024-09-10T20:58:53.904758+0000","last_clean":"2024-09-10T20:58:53.904758+0000","last_became_active":"2024-09-10T20:58:47.245125+0000","last_became_peered":"2024-09-10T20:58:47.245125+0000","last_unstale":"2024-09-10T20:58:53.904758+0000","last_undegraded":"2024-09-10T20:58:53.904758+0000","last_fullsized":"2024-09-10T20:58:53.904758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:29.836234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.259179+0000","last_change":"2024-09-10T20:58:47.242826+0000","last_active":"2024-09-10T20:58:52.259179+0000","last_peered":"2024-09-10T20:58:52.259179+0000","last_clean":"2024-09-10T20:58:52.259179+0000","last_became_active":"2024-09-10T20:58:47.242462+0000","last_became_peered":"2024-09-10T20:58:47.242462+0000","last_unstale":"2024-09-10T20:58:52.259179+0000","last_undegraded":"2024-09-10T20:58:52.259179+0000","last_fullsized":"2024-09-10T20:58:52.259179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:33.221761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255216+0000","last_change":"2024-09-10T20:58:47.236478+0000","last_active":"2024-09-10T20:58:52.255216+0000","last_peered":"2024-09-10T20:58:52.255216+0000","last_clean":"2024-09-10T20:58:52.255216+0000","last_became_active":"2024-09-10T20:58:47.236141+0000","last_became_peered":"2024-09-10T20:58:47.236141+0000","last_unstale":"2024-09-10T20:58:52.255216+0000","last_undegraded":"2024-09-10T20:58:52.255216+0000","last_fullsized":"2024-09-10T20:58:52.255216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:51.719957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258257+0000","last_change":"2024-09-10T20:58:47.233386+0000","last_active":"2024-09-10T20:58:52.258257+0000","last_peered":"2024-09-10T20:58:52.258257+0000","last_clean":"2024-09-10T20:58:52.258257+0000","last_became_active":"2024-09-10T20:58:47.233110+0000","last_became_peered":"2024-09-10T20:58:47.233110+0000","last_unstale":"2024-09-10T20:58:52.258257+0000","last_undegraded":"2024-09-10T20:58:52.258257+0000","last_fullsized":"2024-09-10T20:58:52.258257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:18.950297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255532+0000","last_change":"2024-09-10T20:58:47.236278+0000","last_active":"2024-09-10T20:58:52.255532+0000","last_peered":"2024-09-10T20:58:52.255532+0000","last_clean":"2024-09-10T20:58:52.255532+0000","last_became_active":"2024-09-10T20:58:47.236005+0000","last_became_peered":"2024-09-10T20:58:47.236005+0000","last_unstale":"2024-09-10T20:58:52.255532+0000","last_undegraded":"2024-09-10T20:58:52.255532+0000","last_fullsized":"2024-09-10T20:58:52.255532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:34:54.893698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903878+0000","last_change":"2024-09-10T20:58:47.247778+0000","last_active":"2024-09-10T20:58:53.903878+0000","last_peered":"2024-09-10T20:58:53.903878+0000","last_clean":"2024-09-10T20:58:53.903878+0000","last_became_active":"2024-09-10T20:58:47.247103+0000","last_became_peered":"2024-09-10T20:58:47.247103+0000","last_unstale":"2024-09-10T20:58:53.903878+0000","last_undegraded":"2024-09-10T20:58:53.903878+0000","last_fullsized":"2024-09-10T20:58:53.903878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:42.546277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904334+0000","last_change":"2024-09-10T20:58:47.230471+0000","last_active":"2024-09-10T20:58:53.904334+0000","last_peered":"2024-09-10T20:58:53.904334+0000","last_clean":"2024-09-10T20:58:53.904334+0000","last_became_active":"2024-09-10T20:58:47.230174+0000","last_became_peered":"2024-09-10T20:58:47.230174+0000","last_unstale":"2024-09-10T20:58:53.904334+0000","last_undegraded":"2024-09-10T20:58:53.904334+0000","last_fullsized":"2024-09-10T20:58:53.904334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:54.301892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905155+0000","last_change":"2024-09-10T20:58:47.245889+0000","last_active":"2024-09-10T20:58:53.905155+0000","last_peered":"2024-09-10T20:58:53.905155+0000","last_clean":"2024-09-10T20:58:53.905155+0000","last_became_active":"2024-09-10T20:58:47.245547+0000","last_became_peered":"2024-09-10T20:58:47.245547+0000","last_unstale":"2024-09-10T20:58:53.905155+0000","last_undegraded":"2024-09-10T20:58:53.905155+0000","last_fullsized":"2024-09-10T20:58:53.905155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:11.073447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903659+0000","last_change":"2024-09-10T20:58:47.234594+0000","last_active":"2024-09-10T20:58:53.903659+0000","last_peered":"2024-09-10T20:58:53.903659+0000","last_clean":"2024-09-10T20:58:53.903659+0000","last_became_active":"2024-09-10T20:58:47.234275+0000","last_became_peered":"2024-09-10T20:58:47.234275+0000","last_unstale":"2024-09-10T20:58:53.903659+0000","last_undegraded":"2024-09-10T20:58:53.903659+0000","last_fullsized":"2024-09-10T20:58:53.903659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:25.490881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256889+0000","last_change":"2024-09-10T20:58:47.236653+0000","last_active":"2024-09-10T20:58:52.256889+0000","last_peered":"2024-09-10T20:58:52.256889+0000","last_clean":"2024-09-10T20:58:52.256889+0000","last_became_active":"2024-09-10T20:58:47.236361+0000","last_became_peered":"2024-09-10T20:58:47.236361+0000","last_unstale":"2024-09-10T20:58:52.256889+0000","last_undegraded":"2024-09-10T20:58:52.256889+0000","last_fullsized":"2024-09-10T20:58:52.256889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:50.078953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255103+0000","last_change":"2024-09-10T20:58:47.247779+0000","last_active":"2024-09-10T20:58:52.255103+0000","last_peered":"2024-09-10T20:58:52.255103+0000","last_clean":"2024-09-10T20:58:52.255103+0000","last_became_active":"2024-09-10T20:58:47.246921+0000","last_became_peered":"2024-09-10T20:58:47.246921+0000","last_unstale":"2024-09-10T20:58:52.255103+0000","last_undegraded":"2024-09-10T20:58:52.255103+0000","last_fullsized":"2024-09-10T20:58:52.255103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:07.109711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316370+0000","last_change":"2024-09-10T20:58:47.233169+0000","last_active":"2024-09-10T20:58:53.316370+0000","last_peered":"2024-09-10T20:58:53.316370+0000","last_clean":"2024-09-10T20:58:53.316370+0000","last_became_active":"2024-09-10T20:58:47.232869+0000","last_became_peered":"2024-09-10T20:58:47.232869+0000","last_unstale":"2024-09-10T20:58:53.316370+0000","last_undegraded":"2024-09-10T20:58:53.316370+0000","last_fullsized":"2024-09-10T20:58:53.316370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:37.216505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903723+0000","last_change":"2024-09-10T20:58:47.241778+0000","last_active":"2024-09-10T20:58:53.903723+0000","last_peered":"2024-09-10T20:58:53.903723+0000","last_clean":"2024-09-10T20:58:53.903723+0000","last_became_active":"2024-09-10T20:58:47.241164+0000","last_became_peered":"2024-09-10T20:58:47.241164+0000","last_unstale":"2024-09-10T20:58:53.903723+0000","last_undegraded":"2024-09-10T20:58:53.903723+0000","last_fullsized":"2024-09-10T20:58:53.903723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:59.707834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258272+0000","last_change":"2024-09-10T20:58:47.234831+0000","last_active":"2024-09-10T20:58:52.258272+0000","last_peered":"2024-09-10T20:58:52.258272+0000","last_clean":"2024-09-10T20:58:52.258272+0000","last_became_active":"2024-09-10T20:58:47.234566+0000","last_became_peered":"2024-09-10T20:58:47.234566+0000","last_unstale":"2024-09-10T20:58:52.258272+0000","last_undegraded":"2024-09-10T20:58:52.258272+0000","last_fullsized":"2024-09-10T20:58:52.258272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:29.836234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257006+0000","last_change":"2024-09-10T20:58:47.231297+0000","last_active":"2024-09-10T20:58:52.257006+0000","last_peered":"2024-09-10T20:58:52.257006+0000","last_clean":"2024-09-10T20:58:52.257006+0000","last_became_active":"2024-09-10T20:58:47.230943+0000","last_became_peered":"2024-09-10T20:58:47.230943+0000","last_unstale":"2024-09-10T20:58:52.257006+0000","last_undegraded":"2024-09-10T20:58:52.257006+0000","last_fullsized":"2024-09-10T20:58:52.257006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:29.113846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904210+0000","last_change":"2024-09-10T20:58:47.232404+0000","last_active":"2024-09-10T20:58:53.904210+0000","last_peered":"2024-09-10T20:58:53.904210+0000","last_clean":"2024-09-10T20:58:53.904210+0000","last_became_active":"2024-09-10T20:58:47.232127+0000","last_became_peered":"2024-09-10T20:58:47.232127+0000","last_unstale":"2024-09-10T20:58:53.904210+0000","last_undegraded":"2024-09-10T20:58:53.904210+0000","last_fullsized":"2024-09-10T20:58:53.904210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:39.726666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904756+0000","last_change":"2024-09-10T20:58:47.231816+0000","last_active":"2024-09-10T20:58:53.904756+0000","last_peered":"2024-09-10T20:58:53.904756+0000","last_clean":"2024-09-10T20:58:53.904756+0000","last_became_active":"2024-09-10T20:58:47.231545+0000","last_became_peered":"2024-09-10T20:58:47.231545+0000","last_unstale":"2024-09-10T20:58:53.904756+0000","last_undegraded":"2024-09-10T20:58:53.904756+0000","last_fullsized":"2024-09-10T20:58:53.904756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:54.993134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905112+0000","last_change":"2024-09-10T20:58:47.246656+0000","last_active":"2024-09-10T20:58:53.905112+0000","last_peered":"2024-09-10T20:58:53.905112+0000","last_clean":"2024-09-10T20:58:53.905112+0000","last_became_active":"2024-09-10T20:58:47.245647+0000","last_became_peered":"2024-09-10T20:58:47.245647+0000","last_unstale":"2024-09-10T20:58:53.905112+0000","last_undegraded":"2024-09-10T20:58:53.905112+0000","last_fullsized":"2024-09-10T20:58:53.905112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:35.158790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258006+0000","last_change":"2024-09-10T20:58:47.235102+0000","last_active":"2024-09-10T20:58:52.258006+0000","last_peered":"2024-09-10T20:58:52.258006+0000","last_clean":"2024-09-10T20:58:52.258006+0000","last_became_active":"2024-09-10T20:58:47.233909+0000","last_became_peered":"2024-09-10T20:58:47.233909+0000","last_unstale":"2024-09-10T20:58:52.258006+0000","last_undegraded":"2024-09-10T20:58:52.258006+0000","last_fullsized":"2024-09-10T20:58:52.258006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:59.707834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904152+0000","last_change":"2024-09-10T20:58:47.242793+0000","last_active":"2024-09-10T20:58:53.904152+0000","last_peered":"2024-09-10T20:58:53.904152+0000","last_clean":"2024-09-10T20:58:53.904152+0000","last_became_active":"2024-09-10T20:58:47.242495+0000","last_became_peered":"2024-09-10T20:58:47.242495+0000","last_unstale":"2024-09-10T20:58:53.904152+0000","last_undegraded":"2024-09-10T20:58:53.904152+0000","last_fullsized":"2024-09-10T20:58:53.904152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:21.457962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255428+0000","last_change":"2024-09-10T20:58:47.244668+0000","last_active":"2024-09-10T20:58:52.255428+0000","last_peered":"2024-09-10T20:58:52.255428+0000","last_clean":"2024-09-10T20:58:52.255428+0000","last_became_active":"2024-09-10T20:58:47.244110+0000","last_became_peered":"2024-09-10T20:58:47.244110+0000","last_unstale":"2024-09-10T20:58:52.255428+0000","last_undegraded":"2024-09-10T20:58:52.255428+0000","last_fullsized":"2024-09-10T20:58:52.255428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:41.633550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904570+0000","last_change":"2024-09-10T20:58:47.241409+0000","last_active":"2024-09-10T20:58:53.904570+0000","last_peered":"2024-09-10T20:58:53.904570+0000","last_clean":"2024-09-10T20:58:53.904570+0000","last_became_active":"2024-09-10T20:58:47.241104+0000","last_became_peered":"2024-09-10T20:58:47.241104+0000","last_unstale":"2024-09-10T20:58:53.904570+0000","last_undegraded":"2024-09-10T20:58:53.904570+0000","last_fullsized":"2024-09-10T20:58:53.904570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:08.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255561+0000","last_change":"2024-09-10T20:58:47.246735+0000","last_active":"2024-09-10T20:58:52.255561+0000","last_peered":"2024-09-10T20:58:52.255561+0000","last_clean":"2024-09-10T20:58:52.255561+0000","last_became_active":"2024-09-10T20:58:47.246491+0000","last_became_peered":"2024-09-10T20:58:47.246491+0000","last_unstale":"2024-09-10T20:58:52.255561+0000","last_undegraded":"2024-09-10T20:58:52.255561+0000","last_fullsized":"2024-09-10T20:58:52.255561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:16.591593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904106+0000","last_change":"2024-09-10T20:58:47.246507+0000","last_active":"2024-09-10T20:58:53.904106+0000","last_peered":"2024-09-10T20:58:53.904106+0000","last_clean":"2024-09-10T20:58:53.904106+0000","last_became_active":"2024-09-10T20:58:47.245963+0000","last_became_peered":"2024-09-10T20:58:47.245963+0000","last_unstale":"2024-09-10T20:58:53.904106+0000","last_undegraded":"2024-09-10T20:58:53.904106+0000","last_fullsized":"2024-09-10T20:58:53.904106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:04.498018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316869+0000","last_change":"2024-09-10T20:58:47.242014+0000","last_active":"2024-09-10T20:58:53.316869+0000","last_peered":"2024-09-10T20:58:53.316869+0000","last_clean":"2024-09-10T20:58:53.316869+0000","last_became_active":"2024-09-10T20:58:47.241727+0000","last_became_peered":"2024-09-10T20:58:47.241727+0000","last_unstale":"2024-09-10T20:58:53.316869+0000","last_undegraded":"2024-09-10T20:58:53.316869+0000","last_fullsized":"2024-09-10T20:58:53.316869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:13.462491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904966+0000","last_change":"2024-09-10T20:58:47.246360+0000","last_active":"2024-09-10T20:58:53.904966+0000","last_peered":"2024-09-10T20:58:53.904966+0000","last_clean":"2024-09-10T20:58:53.904966+0000","last_became_active":"2024-09-10T20:58:47.245363+0000","last_became_peered":"2024-09-10T20:58:47.245363+0000","last_unstale":"2024-09-10T20:58:53.904966+0000","last_undegraded":"2024-09-10T20:58:53.904966+0000","last_fullsized":"2024-09-10T20:58:53.904966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:59.710882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":693653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":102829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":102863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":693709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":262144,"data_stored":102793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:59:11.847 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:59:11.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.980+0000 7f2410216640 1 -- 172.21.15.98:0/2004666350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f240815f230 msgr2=0x7f24080b5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.980+0000 7f2410216640 1 --2- 172.21.15.98:0/2004666350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f240815f230 0x7f24080b5950 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2408073b60 tx=0x7f240402a560 comp rx=0 tx=0).stop 2024-09-10T20:59:11.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.980+0000 7f2410216640 1 -- 172.21.15.98:0/2004666350 shutdown_connections 2024-09-10T20:59:11.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.980+0000 7f2410216640 1 --2- 172.21.15.98:0/2004666350 >> v2:172.21.15.190:3300/0 conn(0x7f24080694d0 0x7f24080b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.980+0000 7f2410216640 1 --2- 172.21.15.98:0/2004666350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f240815f230 0x7f24080b5950 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.980+0000 7f2410216640 1 -- 172.21.15.98:0/2004666350 >> 172.21.15.98:0/2004666350 conn(0x7f2408074150 msgr2=0x7f2408074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:11.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.980+0000 7f2410216640 1 -- 172.21.15.98:0/2004666350 shutdown_connections 2024-09-10T20:59:11.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.980+0000 7f2410216640 1 -- 172.21.15.98:0/2004666350 wait complete. 2024-09-10T20:59:11.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.981+0000 7f2410216640 1 Processor -- start 2024-09-10T20:59:11.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.981+0000 7f2410216640 1 -- start start 2024-09-10T20:59:11.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f2410216640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24080694d0 0x7f24081f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f2410216640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24080d02b0 0x7f24080c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f2410216640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f240815f230 0x7f24080c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f2410216640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f24080d0a90 con 0x7f24080d02b0 2024-09-10T20:59:11.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f2410216640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f24080d0910 con 0x7f240815f230 2024-09-10T20:59:11.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f2410216640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f24080d0c10 con 0x7f24080694d0 2024-09-10T20:59:11.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f240d7b1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24080d02b0 0x7f24080c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f240d7b1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24080d02b0 0x7f24080c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55464/0 (socket says 172.21.15.98:55464) 2024-09-10T20:59:11.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f240d7b1640 1 -- 172.21.15.98:0/1517877397 learned_addr learned my addr 172.21.15.98:0/1517877397 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:11.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f240e7b3640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f240815f230 0x7f24080c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f240dfb2640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24080694d0 0x7f24081f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f240d7b1640 1 -- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24080694d0 msgr2=0x7f24081f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f240d7b1640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24080694d0 0x7f24081f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f240d7b1640 1 -- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f240815f230 msgr2=0x7f24080c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:11.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f240d7b1640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f240815f230 0x7f24080c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:11.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f240d7b1640 1 -- 172.21.15.98:0/1517877397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24080c8050 con 0x7f24080d02b0 2024-09-10T20:59:11.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f240e7b3640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f240815f230 0x7f24080c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:11.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.982+0000 7f240d7b1640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24080d02b0 0x7f24080c5fc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f23f800b4d0 tx=0x7f23f800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.983+0000 7f23fe7fc640 1 -- 172.21.15.98:0/1517877397 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8014070 con 0x7f24080d02b0 2024-09-10T20:59:11.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.983+0000 7f23fe7fc640 1 -- 172.21.15.98:0/1517877397 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23f8010960 con 0x7f24080d02b0 2024-09-10T20:59:11.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.983+0000 7f2410216640 1 -- 172.21.15.98:0/1517877397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24080cdef0 con 0x7f24080d02b0 2024-09-10T20:59:11.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.983+0000 7f23fe7fc640 1 -- 172.21.15.98:0/1517877397 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8010db0 con 0x7f24080d02b0 2024-09-10T20:59:11.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.983+0000 7f2410216640 1 -- 172.21.15.98:0/1517877397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24081f7f20 con 0x7f24080d02b0 2024-09-10T20:59:11.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.983+0000 7f2410216640 1 -- 172.21.15.98:0/1517877397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24080d0de0 con 0x7f24080d02b0 2024-09-10T20:59:11.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.984+0000 7f23fe7fc640 1 -- 172.21.15.98:0/1517877397 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f23f800f3c0 con 0x7f24080d02b0 2024-09-10T20:59:11.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.985+0000 7f23fe7fc640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f23ec077f70 0x7f23ec07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:11.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.985+0000 7f23fe7fc640 1 -- 172.21.15.98:0/1517877397 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f23f809a100 con 0x7f24080d02b0 2024-09-10T20:59:11.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.985+0000 7f240dfb2640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f23ec077f70 0x7f23ec07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:11.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.985+0000 7f240dfb2640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f23ec077f70 0x7f23ec07a430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2404033000 tx=0x7f2404004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:11.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:11.989+0000 7f23fe7fc640 1 -- 172.21.15.98:0/1517877397 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f24080d0de0 con 0x7f24080d02b0 2024-09-10T20:59:12.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.158+0000 7f2410216640 1 -- 172.21.15.98:0/1517877397 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f24080c7e00 con 0x7f23ec077f70 2024-09-10T20:59:12.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.164+0000 7f23fe7fc640 1 -- 172.21.15.98:0/1517877397 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f24080c7e00 con 0x7f23ec077f70 2024-09-10T20:59:12.171 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:12.173 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-10T20:59:12.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.167+0000 7f2410216640 1 -- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f23ec077f70 msgr2=0x7f23ec07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:12.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.167+0000 7f2410216640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f23ec077f70 0x7f23ec07a430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2404033000 tx=0x7f2404004730 comp rx=0 tx=0).stop 2024-09-10T20:59:12.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.167+0000 7f2410216640 1 -- 172.21.15.98:0/1517877397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24080d02b0 msgr2=0x7f24080c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:12.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.167+0000 7f2410216640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24080d02b0 0x7f24080c5fc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f23f800b4d0 tx=0x7f23f800b990 comp rx=0 tx=0).stop 2024-09-10T20:59:12.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.167+0000 7f2410216640 1 -- 172.21.15.98:0/1517877397 shutdown_connections 2024-09-10T20:59:12.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.167+0000 7f2410216640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f23ec077f70 0x7f23ec07a430 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.167+0000 7f2410216640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f240815f230 0x7f24080c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.167+0000 7f2410216640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24080d02b0 0x7f24080c5fc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.167+0000 7f2410216640 1 --2- 172.21.15.98:0/1517877397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24080694d0 0x7f24081f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.167+0000 7f2410216640 1 -- 172.21.15.98:0/1517877397 >> 172.21.15.98:0/1517877397 conn(0x7f2408074150 msgr2=0x7f240815d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:12.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.167+0000 7f2410216640 1 -- 172.21.15.98:0/1517877397 shutdown_connections 2024-09-10T20:59:12.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.167+0000 7f2410216640 1 -- 172.21.15.98:0/1517877397 wait complete. 2024-09-10T20:59:12.182 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-09-10T20:59:10.912643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219472,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755248,"statfs":{"total":773094113280,"available":772869373952,"internally_reserved":0,"allocated":3317760,"data_stored":2011640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003089"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904144+0000","last_change":"2024-09-10T20:58:46.222874+0000","last_active":"2024-09-10T20:58:53.904144+0000","last_peered":"2024-09-10T20:58:53.904144+0000","last_clean":"2024-09-10T20:58:53.904144+0000","last_became_active":"2024-09-10T20:58:46.222585+0000","last_became_peered":"2024-09-10T20:58:46.222585+0000","last_unstale":"2024-09-10T20:58:53.904144+0000","last_undegraded":"2024-09-10T20:58:53.904144+0000","last_fullsized":"2024-09-10T20:58:53.904144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:12.557183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258747+0000","last_change":"2024-09-10T20:58:47.236553+0000","last_active":"2024-09-10T20:58:52.258747+0000","last_peered":"2024-09-10T20:58:52.258747+0000","last_clean":"2024-09-10T20:58:52.258747+0000","last_became_active":"2024-09-10T20:58:47.236267+0000","last_became_peered":"2024-09-10T20:58:47.236267+0000","last_unstale":"2024-09-10T20:58:52.258747+0000","last_undegraded":"2024-09-10T20:58:52.258747+0000","last_fullsized":"2024-09-10T20:58:52.258747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:54.301892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258094+0000","last_change":"2024-09-10T20:58:51.557088+0000","last_active":"2024-09-10T20:58:52.258094+0000","last_peered":"2024-09-10T20:58:52.258094+0000","last_clean":"2024-09-10T20:58:52.258094+0000","last_became_active":"2024-09-10T20:58:51.556571+0000","last_became_peered":"2024-09-10T20:58:51.556571+0000","last_unstale":"2024-09-10T20:58:52.258094+0000","last_undegraded":"2024-09-10T20:58:52.258094+0000","last_fullsized":"2024-09-10T20:58:52.258094+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:32.723152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258861+0000","last_change":"2024-09-10T20:58:46.221311+0000","last_active":"2024-09-10T20:58:52.258861+0000","last_peered":"2024-09-10T20:58:52.258861+0000","last_clean":"2024-09-10T20:58:52.258861+0000","last_became_active":"2024-09-10T20:58:46.221053+0000","last_became_peered":"2024-09-10T20:58:46.221053+0000","last_unstale":"2024-09-10T20:58:52.258861+0000","last_undegraded":"2024-09-10T20:58:52.258861+0000","last_fullsized":"2024-09-10T20:58:52.258861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:02.951532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903969+0000","last_change":"2024-09-10T20:58:47.246604+0000","last_active":"2024-09-10T20:58:53.903969+0000","last_peered":"2024-09-10T20:58:53.903969+0000","last_clean":"2024-09-10T20:58:53.903969+0000","last_became_active":"2024-09-10T20:58:47.244966+0000","last_became_peered":"2024-09-10T20:58:47.244966+0000","last_unstale":"2024-09-10T20:58:53.903969+0000","last_undegraded":"2024-09-10T20:58:53.903969+0000","last_fullsized":"2024-09-10T20:58:53.903969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:54.301892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255041+0000","last_change":"2024-09-10T20:58:51.275679+0000","last_active":"2024-09-10T20:58:52.255041+0000","last_peered":"2024-09-10T20:58:52.255041+0000","last_clean":"2024-09-10T20:58:52.255041+0000","last_became_active":"2024-09-10T20:58:51.275227+0000","last_became_peered":"2024-09-10T20:58:51.275227+0000","last_unstale":"2024-09-10T20:58:52.255041+0000","last_undegraded":"2024-09-10T20:58:52.255041+0000","last_fullsized":"2024-09-10T20:58:52.255041+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:07.240517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905109+0000","last_change":"2024-09-10T20:58:46.223303+0000","last_active":"2024-09-10T20:58:53.905109+0000","last_peered":"2024-09-10T20:58:53.905109+0000","last_clean":"2024-09-10T20:58:53.905109+0000","last_became_active":"2024-09-10T20:58:46.222924+0000","last_became_peered":"2024-09-10T20:58:46.222924+0000","last_unstale":"2024-09-10T20:58:53.905109+0000","last_undegraded":"2024-09-10T20:58:53.905109+0000","last_fullsized":"2024-09-10T20:58:53.905109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:37.987408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255679+0000","last_change":"2024-09-10T20:58:47.231063+0000","last_active":"2024-09-10T20:58:52.255679+0000","last_peered":"2024-09-10T20:58:52.255679+0000","last_clean":"2024-09-10T20:58:52.255679+0000","last_became_active":"2024-09-10T20:58:47.230764+0000","last_became_peered":"2024-09-10T20:58:47.230764+0000","last_unstale":"2024-09-10T20:58:52.255679+0000","last_undegraded":"2024-09-10T20:58:52.255679+0000","last_fullsized":"2024-09-10T20:58:52.255679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:17.763154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256830+0000","last_change":"2024-09-10T20:58:51.275019+0000","last_active":"2024-09-10T20:58:52.256830+0000","last_peered":"2024-09-10T20:58:52.256830+0000","last_clean":"2024-09-10T20:58:52.256830+0000","last_became_active":"2024-09-10T20:58:51.274298+0000","last_became_peered":"2024-09-10T20:58:51.274298+0000","last_unstale":"2024-09-10T20:58:52.256830+0000","last_undegraded":"2024-09-10T20:58:52.256830+0000","last_fullsized":"2024-09-10T20:58:52.256830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.486482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904772+0000","last_change":"2024-09-10T20:58:46.231109+0000","last_active":"2024-09-10T20:58:53.904772+0000","last_peered":"2024-09-10T20:58:53.904772+0000","last_clean":"2024-09-10T20:58:53.904772+0000","last_became_active":"2024-09-10T20:58:46.230798+0000","last_became_peered":"2024-09-10T20:58:46.230798+0000","last_unstale":"2024-09-10T20:58:53.904772+0000","last_undegraded":"2024-09-10T20:58:53.904772+0000","last_fullsized":"2024-09-10T20:58:53.904772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:31.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256327+0000","last_change":"2024-09-10T20:58:47.230225+0000","last_active":"2024-09-10T20:58:52.256327+0000","last_peered":"2024-09-10T20:58:52.256327+0000","last_clean":"2024-09-10T20:58:52.256327+0000","last_became_active":"2024-09-10T20:58:47.229943+0000","last_became_peered":"2024-09-10T20:58:47.229943+0000","last_unstale":"2024-09-10T20:58:52.256327+0000","last_undegraded":"2024-09-10T20:58:52.256327+0000","last_fullsized":"2024-09-10T20:58:52.256327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:45.849686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256190+0000","last_change":"2024-09-10T20:58:51.270577+0000","last_active":"2024-09-10T20:58:52.256190+0000","last_peered":"2024-09-10T20:58:52.256190+0000","last_clean":"2024-09-10T20:58:52.256190+0000","last_became_active":"2024-09-10T20:58:51.270223+0000","last_became_peered":"2024-09-10T20:58:51.270223+0000","last_unstale":"2024-09-10T20:58:52.256190+0000","last_undegraded":"2024-09-10T20:58:52.256190+0000","last_fullsized":"2024-09-10T20:58:52.256190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:45.149805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904189+0000","last_change":"2024-09-10T20:58:47.230068+0000","last_active":"2024-09-10T20:58:53.904189+0000","last_peered":"2024-09-10T20:58:53.904189+0000","last_clean":"2024-09-10T20:58:53.904189+0000","last_became_active":"2024-09-10T20:58:47.229755+0000","last_became_peered":"2024-09-10T20:58:47.229755+0000","last_unstale":"2024-09-10T20:58:53.904189+0000","last_undegraded":"2024-09-10T20:58:53.904189+0000","last_fullsized":"2024-09-10T20:58:53.904189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:04.498018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904966+0000","last_change":"2024-09-10T20:58:46.227149+0000","last_active":"2024-09-10T20:58:53.904966+0000","last_peered":"2024-09-10T20:58:53.904966+0000","last_clean":"2024-09-10T20:58:53.904966+0000","last_became_active":"2024-09-10T20:58:46.226863+0000","last_became_peered":"2024-09-10T20:58:46.226863+0000","last_unstale":"2024-09-10T20:58:53.904966+0000","last_undegraded":"2024-09-10T20:58:53.904966+0000","last_fullsized":"2024-09-10T20:58:53.904966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:28.108849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256521+0000","last_change":"2024-09-10T20:58:51.280288+0000","last_active":"2024-09-10T20:58:52.256521+0000","last_peered":"2024-09-10T20:58:52.256521+0000","last_clean":"2024-09-10T20:58:52.256521+0000","last_became_active":"2024-09-10T20:58:51.279918+0000","last_became_peered":"2024-09-10T20:58:51.279918+0000","last_unstale":"2024-09-10T20:58:52.256521+0000","last_undegraded":"2024-09-10T20:58:52.256521+0000","last_fullsized":"2024-09-10T20:58:52.256521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:36.307037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258303+0000","last_change":"2024-09-10T20:58:47.246700+0000","last_active":"2024-09-10T20:58:52.258303+0000","last_peered":"2024-09-10T20:58:52.258303+0000","last_clean":"2024-09-10T20:58:52.258303+0000","last_became_active":"2024-09-10T20:58:47.246456+0000","last_became_peered":"2024-09-10T20:58:47.246456+0000","last_unstale":"2024-09-10T20:58:52.258303+0000","last_undegraded":"2024-09-10T20:58:52.258303+0000","last_fullsized":"2024-09-10T20:58:52.258303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:08.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316714+0000","last_change":"2024-09-10T20:58:46.224904+0000","last_active":"2024-09-10T20:58:53.316714+0000","last_peered":"2024-09-10T20:58:53.316714+0000","last_clean":"2024-09-10T20:58:53.316714+0000","last_became_active":"2024-09-10T20:58:46.224619+0000","last_became_peered":"2024-09-10T20:58:46.224619+0000","last_unstale":"2024-09-10T20:58:53.316714+0000","last_undegraded":"2024-09-10T20:58:53.316714+0000","last_fullsized":"2024-09-10T20:58:53.316714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:34.223018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256285+0000","last_change":"2024-09-10T20:58:51.274096+0000","last_active":"2024-09-10T20:58:52.256285+0000","last_peered":"2024-09-10T20:58:52.256285+0000","last_clean":"2024-09-10T20:58:52.256285+0000","last_became_active":"2024-09-10T20:58:51.273636+0000","last_became_peered":"2024-09-10T20:58:51.273636+0000","last_unstale":"2024-09-10T20:58:52.256285+0000","last_undegraded":"2024-09-10T20:58:52.256285+0000","last_fullsized":"2024-09-10T20:58:52.256285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:05.160808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258649+0000","last_change":"2024-09-10T20:58:47.228381+0000","last_active":"2024-09-10T20:58:52.258649+0000","last_peered":"2024-09-10T20:58:52.258649+0000","last_clean":"2024-09-10T20:58:52.258649+0000","last_became_active":"2024-09-10T20:58:47.228080+0000","last_became_peered":"2024-09-10T20:58:47.228080+0000","last_unstale":"2024-09-10T20:58:52.258649+0000","last_undegraded":"2024-09-10T20:58:52.258649+0000","last_fullsized":"2024-09-10T20:58:52.258649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:39.726666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316453+0000","last_change":"2024-09-10T20:58:46.226903+0000","last_active":"2024-09-10T20:58:53.316453+0000","last_peered":"2024-09-10T20:58:53.316453+0000","last_clean":"2024-09-10T20:58:53.316453+0000","last_became_active":"2024-09-10T20:58:46.226649+0000","last_became_peered":"2024-09-10T20:58:46.226649+0000","last_unstale":"2024-09-10T20:58:53.316453+0000","last_undegraded":"2024-09-10T20:58:53.316453+0000","last_fullsized":"2024-09-10T20:58:53.316453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:14.748746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317126+0000","last_change":"2024-09-10T20:58:51.554471+0000","last_active":"2024-09-10T20:58:53.317126+0000","last_peered":"2024-09-10T20:58:53.317126+0000","last_clean":"2024-09-10T20:58:53.317126+0000","last_became_active":"2024-09-10T20:58:51.554080+0000","last_became_peered":"2024-09-10T20:58:51.554080+0000","last_unstale":"2024-09-10T20:58:53.317126+0000","last_undegraded":"2024-09-10T20:58:53.317126+0000","last_fullsized":"2024-09-10T20:58:53.317126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:24.547698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256775+0000","last_change":"2024-09-10T20:58:47.234236+0000","last_active":"2024-09-10T20:58:52.256775+0000","last_peered":"2024-09-10T20:58:52.256775+0000","last_clean":"2024-09-10T20:58:52.256775+0000","last_became_active":"2024-09-10T20:58:47.233924+0000","last_became_peered":"2024-09-10T20:58:47.233924+0000","last_unstale":"2024-09-10T20:58:52.256775+0000","last_undegraded":"2024-09-10T20:58:52.256775+0000","last_fullsized":"2024-09-10T20:58:52.256775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:38.992405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903782+0000","last_change":"2024-09-10T20:58:46.221112+0000","last_active":"2024-09-10T20:58:53.903782+0000","last_peered":"2024-09-10T20:58:53.903782+0000","last_clean":"2024-09-10T20:58:53.903782+0000","last_became_active":"2024-09-10T20:58:46.220837+0000","last_became_peered":"2024-09-10T20:58:46.220837+0000","last_unstale":"2024-09-10T20:58:53.903782+0000","last_undegraded":"2024-09-10T20:58:53.903782+0000","last_fullsized":"2024-09-10T20:58:53.903782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:27.129152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256427+0000","last_change":"2024-09-10T20:58:51.274965+0000","last_active":"2024-09-10T20:58:52.256427+0000","last_peered":"2024-09-10T20:58:52.256427+0000","last_clean":"2024-09-10T20:58:52.256427+0000","last_became_active":"2024-09-10T20:58:51.274557+0000","last_became_peered":"2024-09-10T20:58:51.274557+0000","last_unstale":"2024-09-10T20:58:52.256427+0000","last_undegraded":"2024-09-10T20:58:52.256427+0000","last_fullsized":"2024-09-10T20:58:52.256427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:16.787797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904100+0000","last_change":"2024-09-10T20:58:47.246893+0000","last_active":"2024-09-10T20:58:53.904100+0000","last_peered":"2024-09-10T20:58:53.904100+0000","last_clean":"2024-09-10T20:58:53.904100+0000","last_became_active":"2024-09-10T20:58:47.245226+0000","last_became_peered":"2024-09-10T20:58:47.245226+0000","last_unstale":"2024-09-10T20:58:53.904100+0000","last_undegraded":"2024-09-10T20:58:53.904100+0000","last_fullsized":"2024-09-10T20:58:53.904100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:58.155371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904645+0000","last_change":"2024-09-10T20:58:46.222772+0000","last_active":"2024-09-10T20:58:53.904645+0000","last_peered":"2024-09-10T20:58:53.904645+0000","last_clean":"2024-09-10T20:58:53.904645+0000","last_became_active":"2024-09-10T20:58:46.222461+0000","last_became_peered":"2024-09-10T20:58:46.222461+0000","last_unstale":"2024-09-10T20:58:53.904645+0000","last_undegraded":"2024-09-10T20:58:53.904645+0000","last_fullsized":"2024-09-10T20:58:53.904645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:28.108849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256282+0000","last_change":"2024-09-10T20:58:51.554529+0000","last_active":"2024-09-10T20:58:52.256282+0000","last_peered":"2024-09-10T20:58:52.256282+0000","last_clean":"2024-09-10T20:58:52.256282+0000","last_became_active":"2024-09-10T20:58:51.554104+0000","last_became_peered":"2024-09-10T20:58:51.554104+0000","last_unstale":"2024-09-10T20:58:52.256282+0000","last_undegraded":"2024-09-10T20:58:52.256282+0000","last_fullsized":"2024-09-10T20:58:52.256282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:18.603248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.259179+0000","last_change":"2024-09-10T20:58:47.249780+0000","last_active":"2024-09-10T20:58:52.259179+0000","last_peered":"2024-09-10T20:58:52.259179+0000","last_clean":"2024-09-10T20:58:52.259179+0000","last_became_active":"2024-09-10T20:58:47.249519+0000","last_became_peered":"2024-09-10T20:58:47.249519+0000","last_unstale":"2024-09-10T20:58:52.259179+0000","last_undegraded":"2024-09-10T20:58:52.259179+0000","last_fullsized":"2024-09-10T20:58:52.259179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:58.169931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316888+0000","last_change":"2024-09-10T20:58:46.225535+0000","last_active":"2024-09-10T20:58:53.316888+0000","last_peered":"2024-09-10T20:58:53.316888+0000","last_clean":"2024-09-10T20:58:53.316888+0000","last_became_active":"2024-09-10T20:58:46.225248+0000","last_became_peered":"2024-09-10T20:58:46.225248+0000","last_unstale":"2024-09-10T20:58:53.316888+0000","last_undegraded":"2024-09-10T20:58:53.316888+0000","last_fullsized":"2024-09-10T20:58:53.316888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:22.880233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258298+0000","last_change":"2024-09-10T20:58:51.559242+0000","last_active":"2024-09-10T20:58:52.258298+0000","last_peered":"2024-09-10T20:58:52.258298+0000","last_clean":"2024-09-10T20:58:52.258298+0000","last_became_active":"2024-09-10T20:58:51.558875+0000","last_became_peered":"2024-09-10T20:58:51.558875+0000","last_unstale":"2024-09-10T20:58:52.258298+0000","last_undegraded":"2024-09-10T20:58:52.258298+0000","last_fullsized":"2024-09-10T20:58:52.258298+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:18.246776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317045+0000","last_change":"2024-09-10T20:58:47.228376+0000","last_active":"2024-09-10T20:58:53.317045+0000","last_peered":"2024-09-10T20:58:53.317045+0000","last_clean":"2024-09-10T20:58:53.317045+0000","last_became_active":"2024-09-10T20:58:47.228086+0000","last_became_peered":"2024-09-10T20:58:47.228086+0000","last_unstale":"2024-09-10T20:58:53.317045+0000","last_undegraded":"2024-09-10T20:58:53.317045+0000","last_fullsized":"2024-09-10T20:58:53.317045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:07.109711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905238+0000","last_change":"2024-09-10T20:58:46.221847+0000","last_active":"2024-09-10T20:58:53.905238+0000","last_peered":"2024-09-10T20:58:53.905238+0000","last_clean":"2024-09-10T20:58:53.905238+0000","last_became_active":"2024-09-10T20:58:46.221571+0000","last_became_peered":"2024-09-10T20:58:46.221571+0000","last_unstale":"2024-09-10T20:58:53.905238+0000","last_undegraded":"2024-09-10T20:58:53.905238+0000","last_fullsized":"2024-09-10T20:58:53.905238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:44.242563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903819+0000","last_change":"2024-09-10T20:58:51.554547+0000","last_active":"2024-09-10T20:58:53.903819+0000","last_peered":"2024-09-10T20:58:53.903819+0000","last_clean":"2024-09-10T20:58:53.903819+0000","last_became_active":"2024-09-10T20:58:51.554197+0000","last_became_peered":"2024-09-10T20:58:51.554197+0000","last_unstale":"2024-09-10T20:58:53.903819+0000","last_undegraded":"2024-09-10T20:58:53.903819+0000","last_fullsized":"2024-09-10T20:58:53.903819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:25.499034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256428+0000","last_change":"2024-09-10T20:58:47.243797+0000","last_active":"2024-09-10T20:58:52.256428+0000","last_peered":"2024-09-10T20:58:52.256428+0000","last_clean":"2024-09-10T20:58:52.256428+0000","last_became_active":"2024-09-10T20:58:47.243539+0000","last_became_peered":"2024-09-10T20:58:47.243539+0000","last_unstale":"2024-09-10T20:58:52.256428+0000","last_undegraded":"2024-09-10T20:58:52.256428+0000","last_fullsized":"2024-09-10T20:58:52.256428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:30.226963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317072+0000","last_change":"2024-09-10T20:58:46.221373+0000","last_active":"2024-09-10T20:58:53.317072+0000","last_peered":"2024-09-10T20:58:53.317072+0000","last_clean":"2024-09-10T20:58:53.317072+0000","last_became_active":"2024-09-10T20:58:46.221104+0000","last_became_peered":"2024-09-10T20:58:46.221104+0000","last_unstale":"2024-09-10T20:58:53.317072+0000","last_undegraded":"2024-09-10T20:58:53.317072+0000","last_fullsized":"2024-09-10T20:58:53.317072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:29.110234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316552+0000","last_change":"2024-09-10T20:58:51.278666+0000","last_active":"2024-09-10T20:58:53.316552+0000","last_peered":"2024-09-10T20:58:53.316552+0000","last_clean":"2024-09-10T20:58:53.316552+0000","last_became_active":"2024-09-10T20:58:51.278033+0000","last_became_peered":"2024-09-10T20:58:51.278033+0000","last_unstale":"2024-09-10T20:58:53.316552+0000","last_undegraded":"2024-09-10T20:58:53.316552+0000","last_fullsized":"2024-09-10T20:58:53.316552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:59.252455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257022+0000","last_change":"2024-09-10T20:58:47.246884+0000","last_active":"2024-09-10T20:58:52.257022+0000","last_peered":"2024-09-10T20:58:52.257022+0000","last_clean":"2024-09-10T20:58:52.257022+0000","last_became_active":"2024-09-10T20:58:47.246617+0000","last_became_peered":"2024-09-10T20:58:47.246617+0000","last_unstale":"2024-09-10T20:58:52.257022+0000","last_undegraded":"2024-09-10T20:58:52.257022+0000","last_fullsized":"2024-09-10T20:58:52.257022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:32.318955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256070+0000","last_change":"2024-09-10T20:58:46.225503+0000","last_active":"2024-09-10T20:58:52.256070+0000","last_peered":"2024-09-10T20:58:52.256070+0000","last_clean":"2024-09-10T20:58:52.256070+0000","last_became_active":"2024-09-10T20:58:46.225088+0000","last_became_peered":"2024-09-10T20:58:46.225088+0000","last_unstale":"2024-09-10T20:58:52.256070+0000","last_undegraded":"2024-09-10T20:58:52.256070+0000","last_fullsized":"2024-09-10T20:58:52.256070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:16:36.394527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317025+0000","last_change":"2024-09-10T20:58:51.277749+0000","last_active":"2024-09-10T20:58:53.317025+0000","last_peered":"2024-09-10T20:58:53.317025+0000","last_clean":"2024-09-10T20:58:53.317025+0000","last_became_active":"2024-09-10T20:58:51.277142+0000","last_became_peered":"2024-09-10T20:58:51.277142+0000","last_unstale":"2024-09-10T20:58:53.317025+0000","last_undegraded":"2024-09-10T20:58:53.317025+0000","last_fullsized":"2024-09-10T20:58:53.317025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:42.740546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316380+0000","last_change":"2024-09-10T20:58:47.228660+0000","last_active":"2024-09-10T20:58:53.316380+0000","last_peered":"2024-09-10T20:58:53.316380+0000","last_clean":"2024-09-10T20:58:53.316380+0000","last_became_active":"2024-09-10T20:58:47.228218+0000","last_became_peered":"2024-09-10T20:58:47.228218+0000","last_unstale":"2024-09-10T20:58:53.316380+0000","last_undegraded":"2024-09-10T20:58:53.316380+0000","last_fullsized":"2024-09-10T20:58:53.316380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:51.280929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903422+0000","last_change":"2024-09-10T20:58:46.221228+0000","last_active":"2024-09-10T20:58:53.903422+0000","last_peered":"2024-09-10T20:58:53.903422+0000","last_clean":"2024-09-10T20:58:53.903422+0000","last_became_active":"2024-09-10T20:58:46.220898+0000","last_became_peered":"2024-09-10T20:58:46.220898+0000","last_unstale":"2024-09-10T20:58:53.903422+0000","last_undegraded":"2024-09-10T20:58:53.903422+0000","last_fullsized":"2024-09-10T20:58:53.903422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:18.355281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.317144+0000","last_change":"2024-09-10T20:58:51.274949+0000","last_active":"2024-09-10T20:58:53.317144+0000","last_peered":"2024-09-10T20:58:53.317144+0000","last_clean":"2024-09-10T20:58:53.317144+0000","last_became_active":"2024-09-10T20:58:51.274579+0000","last_became_peered":"2024-09-10T20:58:51.274579+0000","last_unstale":"2024-09-10T20:58:53.317144+0000","last_undegraded":"2024-09-10T20:58:53.317144+0000","last_fullsized":"2024-09-10T20:58:53.317144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:50.176038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903242+0000","last_change":"2024-09-10T20:58:47.240914+0000","last_active":"2024-09-10T20:58:53.903242+0000","last_peered":"2024-09-10T20:58:53.903242+0000","last_clean":"2024-09-10T20:58:53.903242+0000","last_became_active":"2024-09-10T20:58:47.240250+0000","last_became_peered":"2024-09-10T20:58:47.240250+0000","last_unstale":"2024-09-10T20:58:53.903242+0000","last_undegraded":"2024-09-10T20:58:53.903242+0000","last_fullsized":"2024-09-10T20:58:53.903242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:46.184593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903948+0000","last_change":"2024-09-10T20:58:46.224644+0000","last_active":"2024-09-10T20:58:53.903948+0000","last_peered":"2024-09-10T20:58:53.903948+0000","last_clean":"2024-09-10T20:58:53.903948+0000","last_became_active":"2024-09-10T20:58:46.224329+0000","last_became_peered":"2024-09-10T20:58:46.224329+0000","last_unstale":"2024-09-10T20:58:53.903948+0000","last_undegraded":"2024-09-10T20:58:53.903948+0000","last_fullsized":"2024-09-10T20:58:53.903948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:01.590603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904042+0000","last_change":"2024-09-10T20:58:51.554468+0000","last_active":"2024-09-10T20:58:53.904042+0000","last_peered":"2024-09-10T20:58:53.904042+0000","last_clean":"2024-09-10T20:58:53.904042+0000","last_became_active":"2024-09-10T20:58:51.554092+0000","last_became_peered":"2024-09-10T20:58:51.554092+0000","last_unstale":"2024-09-10T20:58:53.904042+0000","last_undegraded":"2024-09-10T20:58:53.904042+0000","last_fullsized":"2024-09-10T20:58:53.904042+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:48.958938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258417+0000","last_change":"2024-09-10T20:58:47.242758+0000","last_active":"2024-09-10T20:58:52.258417+0000","last_peered":"2024-09-10T20:58:52.258417+0000","last_clean":"2024-09-10T20:58:52.258417+0000","last_became_active":"2024-09-10T20:58:47.242439+0000","last_became_peered":"2024-09-10T20:58:47.242439+0000","last_unstale":"2024-09-10T20:58:52.258417+0000","last_undegraded":"2024-09-10T20:58:52.258417+0000","last_fullsized":"2024-09-10T20:58:52.258417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:45.212911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255997+0000","last_change":"2024-09-10T20:58:46.222264+0000","last_active":"2024-09-10T20:58:52.255997+0000","last_peered":"2024-09-10T20:58:52.255997+0000","last_clean":"2024-09-10T20:58:52.255997+0000","last_became_active":"2024-09-10T20:58:46.221414+0000","last_became_peered":"2024-09-10T20:58:46.221414+0000","last_unstale":"2024-09-10T20:58:52.255997+0000","last_undegraded":"2024-09-10T20:58:52.255997+0000","last_fullsized":"2024-09-10T20:58:52.255997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:22.019957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":119,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904053+0000","last_change":"2024-09-10T20:58:36.895993+0000","last_active":"2024-09-10T20:58:53.904053+0000","last_peered":"2024-09-10T20:58:53.904053+0000","last_clean":"2024-09-10T20:58:53.904053+0000","last_became_active":"2024-09-10T20:58:36.894988+0000","last_became_peered":"2024-09-10T20:58:36.894988+0000","last_unstale":"2024-09-10T20:58:53.904053+0000","last_undegraded":"2024-09-10T20:58:53.904053+0000","last_fullsized":"2024-09-10T20:58:53.904053+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_clean_scrub_stamp":"2024-09-10T20:58:35.588702+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.082528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904143+0000","last_change":"2024-09-10T20:58:51.556599+0000","last_active":"2024-09-10T20:58:53.904143+0000","last_peered":"2024-09-10T20:58:53.904143+0000","last_clean":"2024-09-10T20:58:53.904143+0000","last_became_active":"2024-09-10T20:58:51.556210+0000","last_became_peered":"2024-09-10T20:58:51.556210+0000","last_unstale":"2024-09-10T20:58:53.904143+0000","last_undegraded":"2024-09-10T20:58:53.904143+0000","last_fullsized":"2024-09-10T20:58:53.904143+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:36.307037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904986+0000","last_change":"2024-09-10T20:58:47.245638+0000","last_active":"2024-09-10T20:58:53.904986+0000","last_peered":"2024-09-10T20:58:53.904986+0000","last_clean":"2024-09-10T20:58:53.904986+0000","last_became_active":"2024-09-10T20:58:47.244988+0000","last_became_peered":"2024-09-10T20:58:47.244988+0000","last_unstale":"2024-09-10T20:58:53.904986+0000","last_undegraded":"2024-09-10T20:58:53.904986+0000","last_fullsized":"2024-09-10T20:58:53.904986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:04.445410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903834+0000","last_change":"2024-09-10T20:58:46.228090+0000","last_active":"2024-09-10T20:58:53.903834+0000","last_peered":"2024-09-10T20:58:53.903834+0000","last_clean":"2024-09-10T20:58:53.903834+0000","last_became_active":"2024-09-10T20:58:46.227789+0000","last_became_peered":"2024-09-10T20:58:46.227789+0000","last_unstale":"2024-09-10T20:58:53.903834+0000","last_undegraded":"2024-09-10T20:58:53.903834+0000","last_fullsized":"2024-09-10T20:58:53.903834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:37.064288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904078+0000","last_change":"2024-09-10T20:58:51.554553+0000","last_active":"2024-09-10T20:58:53.904078+0000","last_peered":"2024-09-10T20:58:53.904078+0000","last_clean":"2024-09-10T20:58:53.904078+0000","last_became_active":"2024-09-10T20:58:51.554131+0000","last_became_peered":"2024-09-10T20:58:51.554131+0000","last_unstale":"2024-09-10T20:58:53.904078+0000","last_undegraded":"2024-09-10T20:58:53.904078+0000","last_fullsized":"2024-09-10T20:58:53.904078+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:18.603248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255727+0000","last_change":"2024-09-10T20:58:47.245802+0000","last_active":"2024-09-10T20:58:52.255727+0000","last_peered":"2024-09-10T20:58:52.255727+0000","last_clean":"2024-09-10T20:58:52.255727+0000","last_became_active":"2024-09-10T20:58:47.245543+0000","last_became_peered":"2024-09-10T20:58:47.245543+0000","last_unstale":"2024-09-10T20:58:52.255727+0000","last_undegraded":"2024-09-10T20:58:52.255727+0000","last_fullsized":"2024-09-10T20:58:52.255727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:17.787548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316844+0000","last_change":"2024-09-10T20:58:46.226841+0000","last_active":"2024-09-10T20:58:53.316844+0000","last_peered":"2024-09-10T20:58:53.316844+0000","last_clean":"2024-09-10T20:58:53.316844+0000","last_became_active":"2024-09-10T20:58:46.226610+0000","last_became_peered":"2024-09-10T20:58:46.226610+0000","last_unstale":"2024-09-10T20:58:53.316844+0000","last_undegraded":"2024-09-10T20:58:53.316844+0000","last_fullsized":"2024-09-10T20:58:53.316844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:41.552153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904125+0000","last_change":"2024-09-10T20:58:51.274667+0000","last_active":"2024-09-10T20:58:53.904125+0000","last_peered":"2024-09-10T20:58:53.904125+0000","last_clean":"2024-09-10T20:58:53.904125+0000","last_became_active":"2024-09-10T20:58:51.274284+0000","last_became_peered":"2024-09-10T20:58:51.274284+0000","last_unstale":"2024-09-10T20:58:53.904125+0000","last_undegraded":"2024-09-10T20:58:53.904125+0000","last_fullsized":"2024-09-10T20:58:53.904125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:36.498874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905137+0000","last_change":"2024-09-10T20:58:47.245680+0000","last_active":"2024-09-10T20:58:53.905137+0000","last_peered":"2024-09-10T20:58:53.905137+0000","last_clean":"2024-09-10T20:58:53.905137+0000","last_became_active":"2024-09-10T20:58:47.245058+0000","last_became_peered":"2024-09-10T20:58:47.245058+0000","last_unstale":"2024-09-10T20:58:53.905137+0000","last_undegraded":"2024-09-10T20:58:53.905137+0000","last_fullsized":"2024-09-10T20:58:53.905137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:36.576513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903742+0000","last_change":"2024-09-10T20:58:46.222762+0000","last_active":"2024-09-10T20:58:53.903742+0000","last_peered":"2024-09-10T20:58:53.903742+0000","last_clean":"2024-09-10T20:58:53.903742+0000","last_became_active":"2024-09-10T20:58:46.222478+0000","last_became_peered":"2024-09-10T20:58:46.222478+0000","last_unstale":"2024-09-10T20:58:53.903742+0000","last_undegraded":"2024-09-10T20:58:53.903742+0000","last_fullsized":"2024-09-10T20:58:53.903742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:20.646322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256423+0000","last_change":"2024-09-10T20:58:51.554839+0000","last_active":"2024-09-10T20:58:52.256423+0000","last_peered":"2024-09-10T20:58:52.256423+0000","last_clean":"2024-09-10T20:58:52.256423+0000","last_became_active":"2024-09-10T20:58:51.554371+0000","last_became_peered":"2024-09-10T20:58:51.554371+0000","last_unstale":"2024-09-10T20:58:52.256423+0000","last_undegraded":"2024-09-10T20:58:52.256423+0000","last_fullsized":"2024-09-10T20:58:52.256423+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:16.242767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905199+0000","last_change":"2024-09-10T20:58:47.239517+0000","last_active":"2024-09-10T20:58:53.905199+0000","last_peered":"2024-09-10T20:58:53.905199+0000","last_clean":"2024-09-10T20:58:53.905199+0000","last_became_active":"2024-09-10T20:58:47.239275+0000","last_became_peered":"2024-09-10T20:58:47.239275+0000","last_unstale":"2024-09-10T20:58:53.905199+0000","last_undegraded":"2024-09-10T20:58:53.905199+0000","last_fullsized":"2024-09-10T20:58:53.905199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:27.143705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256315+0000","last_change":"2024-09-10T20:58:46.226170+0000","last_active":"2024-09-10T20:58:52.256315+0000","last_peered":"2024-09-10T20:58:52.256315+0000","last_clean":"2024-09-10T20:58:52.256315+0000","last_became_active":"2024-09-10T20:58:46.225888+0000","last_became_peered":"2024-09-10T20:58:46.225888+0000","last_unstale":"2024-09-10T20:58:52.256315+0000","last_undegraded":"2024-09-10T20:58:52.256315+0000","last_fullsized":"2024-09-10T20:58:52.256315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:27.108377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255461+0000","last_change":"2024-09-10T20:58:51.276340+0000","last_active":"2024-09-10T20:58:52.255461+0000","last_peered":"2024-09-10T20:58:52.255461+0000","last_clean":"2024-09-10T20:58:52.255461+0000","last_became_active":"2024-09-10T20:58:51.275903+0000","last_became_peered":"2024-09-10T20:58:51.275903+0000","last_unstale":"2024-09-10T20:58:52.255461+0000","last_undegraded":"2024-09-10T20:58:52.255461+0000","last_fullsized":"2024-09-10T20:58:52.255461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:04.361773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904252+0000","last_change":"2024-09-10T20:58:47.237276+0000","last_active":"2024-09-10T20:58:53.904252+0000","last_peered":"2024-09-10T20:58:53.904252+0000","last_clean":"2024-09-10T20:58:53.904252+0000","last_became_active":"2024-09-10T20:58:47.235334+0000","last_became_peered":"2024-09-10T20:58:47.235334+0000","last_unstale":"2024-09-10T20:58:53.904252+0000","last_undegraded":"2024-09-10T20:58:53.904252+0000","last_fullsized":"2024-09-10T20:58:53.904252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:58.169931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903190+0000","last_change":"2024-09-10T20:58:46.224350+0000","last_active":"2024-09-10T20:58:53.903190+0000","last_peered":"2024-09-10T20:58:53.903190+0000","last_clean":"2024-09-10T20:58:53.903190+0000","last_became_active":"2024-09-10T20:58:46.223931+0000","last_became_peered":"2024-09-10T20:58:46.223931+0000","last_unstale":"2024-09-10T20:58:53.903190+0000","last_undegraded":"2024-09-10T20:58:53.903190+0000","last_fullsized":"2024-09-10T20:58:53.903190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:22.019957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903918+0000","last_change":"2024-09-10T20:58:51.270969+0000","last_active":"2024-09-10T20:58:53.903918+0000","last_peered":"2024-09-10T20:58:53.903918+0000","last_clean":"2024-09-10T20:58:53.903918+0000","last_became_active":"2024-09-10T20:58:51.270229+0000","last_became_peered":"2024-09-10T20:58:51.270229+0000","last_unstale":"2024-09-10T20:58:53.903918+0000","last_undegraded":"2024-09-10T20:58:53.903918+0000","last_fullsized":"2024-09-10T20:58:53.903918+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:35.803700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903940+0000","last_change":"2024-09-10T20:58:47.247096+0000","last_active":"2024-09-10T20:58:53.903940+0000","last_peered":"2024-09-10T20:58:53.903940+0000","last_clean":"2024-09-10T20:58:53.903940+0000","last_became_active":"2024-09-10T20:58:47.246797+0000","last_became_peered":"2024-09-10T20:58:47.246797+0000","last_unstale":"2024-09-10T20:58:53.903940+0000","last_undegraded":"2024-09-10T20:58:53.903940+0000","last_fullsized":"2024-09-10T20:58:53.903940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:18.950297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904318+0000","last_change":"2024-09-10T20:58:46.226890+0000","last_active":"2024-09-10T20:58:53.904318+0000","last_peered":"2024-09-10T20:58:53.904318+0000","last_clean":"2024-09-10T20:58:53.904318+0000","last_became_active":"2024-09-10T20:58:46.226615+0000","last_became_peered":"2024-09-10T20:58:46.226615+0000","last_unstale":"2024-09-10T20:58:53.904318+0000","last_undegraded":"2024-09-10T20:58:53.904318+0000","last_fullsized":"2024-09-10T20:58:53.904318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:08.671158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.259085+0000","last_change":"2024-09-10T20:58:51.556517+0000","last_active":"2024-09-10T20:58:52.259085+0000","last_peered":"2024-09-10T20:58:52.259085+0000","last_clean":"2024-09-10T20:58:52.259085+0000","last_became_active":"2024-09-10T20:58:51.556152+0000","last_became_peered":"2024-09-10T20:58:51.556152+0000","last_unstale":"2024-09-10T20:58:52.259085+0000","last_undegraded":"2024-09-10T20:58:52.259085+0000","last_fullsized":"2024-09-10T20:58:52.259085+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:46.913894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904720+0000","last_change":"2024-09-10T20:58:47.245297+0000","last_active":"2024-09-10T20:58:53.904720+0000","last_peered":"2024-09-10T20:58:53.904720+0000","last_clean":"2024-09-10T20:58:53.904720+0000","last_became_active":"2024-09-10T20:58:47.244947+0000","last_became_peered":"2024-09-10T20:58:47.244947+0000","last_unstale":"2024-09-10T20:58:53.904720+0000","last_undegraded":"2024-09-10T20:58:53.904720+0000","last_fullsized":"2024-09-10T20:58:53.904720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:25.115061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255795+0000","last_change":"2024-09-10T20:58:46.229169+0000","last_active":"2024-09-10T20:58:52.255795+0000","last_peered":"2024-09-10T20:58:52.255795+0000","last_clean":"2024-09-10T20:58:52.255795+0000","last_became_active":"2024-09-10T20:58:46.228622+0000","last_became_peered":"2024-09-10T20:58:46.228622+0000","last_unstale":"2024-09-10T20:58:52.255795+0000","last_undegraded":"2024-09-10T20:58:52.255795+0000","last_fullsized":"2024-09-10T20:58:52.255795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:35.810342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258536+0000","last_change":"2024-09-10T20:58:51.554436+0000","last_active":"2024-09-10T20:58:52.258536+0000","last_peered":"2024-09-10T20:58:52.258536+0000","last_clean":"2024-09-10T20:58:52.258536+0000","last_became_active":"2024-09-10T20:58:51.554038+0000","last_became_peered":"2024-09-10T20:58:51.554038+0000","last_unstale":"2024-09-10T20:58:52.258536+0000","last_undegraded":"2024-09-10T20:58:52.258536+0000","last_fullsized":"2024-09-10T20:58:52.258536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:33.352140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903833+0000","last_change":"2024-09-10T20:58:47.247094+0000","last_active":"2024-09-10T20:58:53.903833+0000","last_peered":"2024-09-10T20:58:53.903833+0000","last_clean":"2024-09-10T20:58:53.903833+0000","last_became_active":"2024-09-10T20:58:47.246095+0000","last_became_peered":"2024-09-10T20:58:47.246095+0000","last_unstale":"2024-09-10T20:58:53.903833+0000","last_undegraded":"2024-09-10T20:58:53.903833+0000","last_fullsized":"2024-09-10T20:58:53.903833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:39.726666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904184+0000","last_change":"2024-09-10T20:58:46.226623+0000","last_active":"2024-09-10T20:58:53.904184+0000","last_peered":"2024-09-10T20:58:53.904184+0000","last_clean":"2024-09-10T20:58:53.904184+0000","last_became_active":"2024-09-10T20:58:46.226326+0000","last_became_peered":"2024-09-10T20:58:46.226326+0000","last_unstale":"2024-09-10T20:58:53.904184+0000","last_undegraded":"2024-09-10T20:58:53.904184+0000","last_fullsized":"2024-09-10T20:58:53.904184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:02:21.934804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256251+0000","last_change":"2024-09-10T20:58:51.556678+0000","last_active":"2024-09-10T20:58:52.256251+0000","last_peered":"2024-09-10T20:58:52.256251+0000","last_clean":"2024-09-10T20:58:52.256251+0000","last_became_active":"2024-09-10T20:58:51.556271+0000","last_became_peered":"2024-09-10T20:58:51.556271+0000","last_unstale":"2024-09-10T20:58:52.256251+0000","last_undegraded":"2024-09-10T20:58:52.256251+0000","last_fullsized":"2024-09-10T20:58:52.256251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:02.438361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904916+0000","last_change":"2024-09-10T20:58:47.243793+0000","last_active":"2024-09-10T20:58:53.904916+0000","last_peered":"2024-09-10T20:58:53.904916+0000","last_clean":"2024-09-10T20:58:53.904916+0000","last_became_active":"2024-09-10T20:58:47.243537+0000","last_became_peered":"2024-09-10T20:58:47.243537+0000","last_unstale":"2024-09-10T20:58:53.904916+0000","last_undegraded":"2024-09-10T20:58:53.904916+0000","last_fullsized":"2024-09-10T20:58:53.904916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:04.464193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:56.612638+0000","last_change":"2024-09-10T20:58:46.225562+0000","last_active":"2024-09-10T20:58:56.612638+0000","last_peered":"2024-09-10T20:58:56.612638+0000","last_clean":"2024-09-10T20:58:56.612638+0000","last_became_active":"2024-09-10T20:58:46.225260+0000","last_became_peered":"2024-09-10T20:58:46.225260+0000","last_unstale":"2024-09-10T20:58:56.612638+0000","last_undegraded":"2024-09-10T20:58:56.612638+0000","last_fullsized":"2024-09-10T20:58:56.612638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:31.313957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316695+0000","last_change":"2024-09-10T20:58:51.278993+0000","last_active":"2024-09-10T20:58:53.316695+0000","last_peered":"2024-09-10T20:58:53.316695+0000","last_clean":"2024-09-10T20:58:53.316695+0000","last_became_active":"2024-09-10T20:58:51.278069+0000","last_became_peered":"2024-09-10T20:58:51.278069+0000","last_unstale":"2024-09-10T20:58:53.316695+0000","last_undegraded":"2024-09-10T20:58:53.316695+0000","last_fullsized":"2024-09-10T20:58:53.316695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:21.828620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257045+0000","last_change":"2024-09-10T20:58:47.235717+0000","last_active":"2024-09-10T20:58:52.257045+0000","last_peered":"2024-09-10T20:58:52.257045+0000","last_clean":"2024-09-10T20:58:52.257045+0000","last_became_active":"2024-09-10T20:58:47.235451+0000","last_became_peered":"2024-09-10T20:58:47.235451+0000","last_unstale":"2024-09-10T20:58:52.257045+0000","last_undegraded":"2024-09-10T20:58:52.257045+0000","last_fullsized":"2024-09-10T20:58:52.257045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:28.134150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316771+0000","last_change":"2024-09-10T20:58:46.226035+0000","last_active":"2024-09-10T20:58:53.316771+0000","last_peered":"2024-09-10T20:58:53.316771+0000","last_clean":"2024-09-10T20:58:53.316771+0000","last_became_active":"2024-09-10T20:58:46.225778+0000","last_became_peered":"2024-09-10T20:58:46.225778+0000","last_unstale":"2024-09-10T20:58:53.316771+0000","last_undegraded":"2024-09-10T20:58:53.316771+0000","last_fullsized":"2024-09-10T20:58:53.316771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:57.175951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256534+0000","last_change":"2024-09-10T20:58:51.274974+0000","last_active":"2024-09-10T20:58:52.256534+0000","last_peered":"2024-09-10T20:58:52.256534+0000","last_clean":"2024-09-10T20:58:52.256534+0000","last_became_active":"2024-09-10T20:58:51.274597+0000","last_became_peered":"2024-09-10T20:58:51.274597+0000","last_unstale":"2024-09-10T20:58:52.256534+0000","last_undegraded":"2024-09-10T20:58:52.256534+0000","last_fullsized":"2024-09-10T20:58:52.256534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:31.184777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904058+0000","last_change":"2024-09-10T20:58:47.245724+0000","last_active":"2024-09-10T20:58:53.904058+0000","last_peered":"2024-09-10T20:58:53.904058+0000","last_clean":"2024-09-10T20:58:53.904058+0000","last_became_active":"2024-09-10T20:58:47.245252+0000","last_became_peered":"2024-09-10T20:58:47.245252+0000","last_unstale":"2024-09-10T20:58:53.904058+0000","last_undegraded":"2024-09-10T20:58:53.904058+0000","last_fullsized":"2024-09-10T20:58:53.904058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:29.836234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255635+0000","last_change":"2024-09-10T20:58:46.229938+0000","last_active":"2024-09-10T20:58:52.255635+0000","last_peered":"2024-09-10T20:58:52.255635+0000","last_clean":"2024-09-10T20:58:52.255635+0000","last_became_active":"2024-09-10T20:58:46.229121+0000","last_became_peered":"2024-09-10T20:58:46.229121+0000","last_unstale":"2024-09-10T20:58:52.255635+0000","last_undegraded":"2024-09-10T20:58:52.255635+0000","last_fullsized":"2024-09-10T20:58:52.255635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:18.355281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258591+0000","last_change":"2024-09-10T20:58:51.280436+0000","last_active":"2024-09-10T20:58:52.258591+0000","last_peered":"2024-09-10T20:58:52.258591+0000","last_clean":"2024-09-10T20:58:52.258591+0000","last_became_active":"2024-09-10T20:58:51.279981+0000","last_became_peered":"2024-09-10T20:58:51.279981+0000","last_unstale":"2024-09-10T20:58:52.258591+0000","last_undegraded":"2024-09-10T20:58:52.258591+0000","last_fullsized":"2024-09-10T20:58:52.258591+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:48:01.843001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255541+0000","last_change":"2024-09-10T20:58:47.235938+0000","last_active":"2024-09-10T20:58:52.255541+0000","last_peered":"2024-09-10T20:58:52.255541+0000","last_clean":"2024-09-10T20:58:52.255541+0000","last_became_active":"2024-09-10T20:58:47.235628+0000","last_became_peered":"2024-09-10T20:58:47.235628+0000","last_unstale":"2024-09-10T20:58:52.255541+0000","last_undegraded":"2024-09-10T20:58:52.255541+0000","last_fullsized":"2024-09-10T20:58:52.255541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:13.462491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904541+0000","last_change":"2024-09-10T20:58:46.230618+0000","last_active":"2024-09-10T20:58:53.904541+0000","last_peered":"2024-09-10T20:58:53.904541+0000","last_clean":"2024-09-10T20:58:53.904541+0000","last_became_active":"2024-09-10T20:58:46.230289+0000","last_became_peered":"2024-09-10T20:58:46.230289+0000","last_unstale":"2024-09-10T20:58:53.904541+0000","last_undegraded":"2024-09-10T20:58:53.904541+0000","last_fullsized":"2024-09-10T20:58:53.904541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:51.217167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258193+0000","last_change":"2024-09-10T20:58:51.557495+0000","last_active":"2024-09-10T20:58:52.258193+0000","last_peered":"2024-09-10T20:58:52.258193+0000","last_clean":"2024-09-10T20:58:52.258193+0000","last_became_active":"2024-09-10T20:58:51.556492+0000","last_became_peered":"2024-09-10T20:58:51.556492+0000","last_unstale":"2024-09-10T20:58:52.258193+0000","last_undegraded":"2024-09-10T20:58:52.258193+0000","last_fullsized":"2024-09-10T20:58:52.258193+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:48.716121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903643+0000","last_change":"2024-09-10T20:58:51.555588+0000","last_active":"2024-09-10T20:58:53.903643+0000","last_peered":"2024-09-10T20:58:53.903643+0000","last_clean":"2024-09-10T20:58:53.903643+0000","last_became_active":"2024-09-10T20:58:51.555232+0000","last_became_peered":"2024-09-10T20:58:51.555232+0000","last_unstale":"2024-09-10T20:58:53.903643+0000","last_undegraded":"2024-09-10T20:58:53.903643+0000","last_fullsized":"2024-09-10T20:58:53.903643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.486482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316315+0000","last_change":"2024-09-10T20:58:47.239189+0000","last_active":"2024-09-10T20:58:53.316315+0000","last_peered":"2024-09-10T20:58:53.316315+0000","last_clean":"2024-09-10T20:58:53.316315+0000","last_became_active":"2024-09-10T20:58:47.238926+0000","last_became_peered":"2024-09-10T20:58:47.238926+0000","last_unstale":"2024-09-10T20:58:53.316315+0000","last_undegraded":"2024-09-10T20:58:53.316315+0000","last_fullsized":"2024-09-10T20:58:53.316315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:02.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255350+0000","last_change":"2024-09-10T20:58:46.229369+0000","last_active":"2024-09-10T20:58:52.255350+0000","last_peered":"2024-09-10T20:58:52.255350+0000","last_clean":"2024-09-10T20:58:52.255350+0000","last_became_active":"2024-09-10T20:58:46.229094+0000","last_became_peered":"2024-09-10T20:58:46.229094+0000","last_unstale":"2024-09-10T20:58:52.255350+0000","last_undegraded":"2024-09-10T20:58:52.255350+0000","last_fullsized":"2024-09-10T20:58:52.255350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:20.646322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904390+0000","last_change":"2024-09-10T20:58:51.280161+0000","last_active":"2024-09-10T20:58:53.904390+0000","last_peered":"2024-09-10T20:58:53.904390+0000","last_clean":"2024-09-10T20:58:53.904390+0000","last_became_active":"2024-09-10T20:58:51.279714+0000","last_became_peered":"2024-09-10T20:58:51.279714+0000","last_unstale":"2024-09-10T20:58:53.904390+0000","last_undegraded":"2024-09-10T20:58:53.904390+0000","last_fullsized":"2024-09-10T20:58:53.904390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.486482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258577+0000","last_change":"2024-09-10T20:58:47.248033+0000","last_active":"2024-09-10T20:58:52.258577+0000","last_peered":"2024-09-10T20:58:52.258577+0000","last_clean":"2024-09-10T20:58:52.258577+0000","last_became_active":"2024-09-10T20:58:47.247761+0000","last_became_peered":"2024-09-10T20:58:47.247761+0000","last_unstale":"2024-09-10T20:58:52.258577+0000","last_undegraded":"2024-09-10T20:58:52.258577+0000","last_fullsized":"2024-09-10T20:58:52.258577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:04.498018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258357+0000","last_change":"2024-09-10T20:58:46.223392+0000","last_active":"2024-09-10T20:58:52.258357+0000","last_peered":"2024-09-10T20:58:52.258357+0000","last_clean":"2024-09-10T20:58:52.258357+0000","last_became_active":"2024-09-10T20:58:46.223123+0000","last_became_peered":"2024-09-10T20:58:46.223123+0000","last_unstale":"2024-09-10T20:58:52.258357+0000","last_undegraded":"2024-09-10T20:58:52.258357+0000","last_fullsized":"2024-09-10T20:58:52.258357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:16.570128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904433+0000","last_change":"2024-09-10T20:58:46.227651+0000","last_active":"2024-09-10T20:58:53.904433+0000","last_peered":"2024-09-10T20:58:53.904433+0000","last_clean":"2024-09-10T20:58:53.904433+0000","last_became_active":"2024-09-10T20:58:46.226418+0000","last_became_peered":"2024-09-10T20:58:46.226418+0000","last_unstale":"2024-09-10T20:58:53.904433+0000","last_undegraded":"2024-09-10T20:58:53.904433+0000","last_fullsized":"2024-09-10T20:58:53.904433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:02.951532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904158+0000","last_change":"2024-09-10T20:58:47.245394+0000","last_active":"2024-09-10T20:58:53.904158+0000","last_peered":"2024-09-10T20:58:53.904158+0000","last_clean":"2024-09-10T20:58:53.904158+0000","last_became_active":"2024-09-10T20:58:47.243973+0000","last_became_peered":"2024-09-10T20:58:47.243973+0000","last_unstale":"2024-09-10T20:58:53.904158+0000","last_undegraded":"2024-09-10T20:58:53.904158+0000","last_fullsized":"2024-09-10T20:58:53.904158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:54.993134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903643+0000","last_change":"2024-09-10T20:58:51.274539+0000","last_active":"2024-09-10T20:58:53.903643+0000","last_peered":"2024-09-10T20:58:53.903643+0000","last_clean":"2024-09-10T20:58:53.903643+0000","last_became_active":"2024-09-10T20:58:51.274124+0000","last_became_peered":"2024-09-10T20:58:51.274124+0000","last_unstale":"2024-09-10T20:58:53.903643+0000","last_undegraded":"2024-09-10T20:58:53.903643+0000","last_fullsized":"2024-09-10T20:58:53.903643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:20.469415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258431+0000","last_change":"2024-09-10T20:58:46.223353+0000","last_active":"2024-09-10T20:58:52.258431+0000","last_peered":"2024-09-10T20:58:52.258431+0000","last_clean":"2024-09-10T20:58:52.258431+0000","last_became_active":"2024-09-10T20:58:46.223059+0000","last_became_peered":"2024-09-10T20:58:46.223059+0000","last_unstale":"2024-09-10T20:58:52.258431+0000","last_undegraded":"2024-09-10T20:58:52.258431+0000","last_fullsized":"2024-09-10T20:58:52.258431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:45.198358+0000","last_clean_scrub_stamp":"2024-09-10T20:58:45.198358+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:02:21.934804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257112+0000","last_change":"2024-09-10T20:58:47.233445+0000","last_active":"2024-09-10T20:58:52.257112+0000","last_peered":"2024-09-10T20:58:52.257112+0000","last_clean":"2024-09-10T20:58:52.257112+0000","last_became_active":"2024-09-10T20:58:47.233157+0000","last_became_peered":"2024-09-10T20:58:47.233157+0000","last_unstale":"2024-09-10T20:58:52.257112+0000","last_undegraded":"2024-09-10T20:58:52.257112+0000","last_fullsized":"2024-09-10T20:58:52.257112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:52.222165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258673+0000","last_change":"2024-09-10T20:58:51.281498+0000","last_active":"2024-09-10T20:58:52.258673+0000","last_peered":"2024-09-10T20:58:52.258673+0000","last_clean":"2024-09-10T20:58:52.258673+0000","last_became_active":"2024-09-10T20:58:51.281027+0000","last_became_peered":"2024-09-10T20:58:51.281027+0000","last_unstale":"2024-09-10T20:58:52.258673+0000","last_undegraded":"2024-09-10T20:58:52.258673+0000","last_fullsized":"2024-09-10T20:58:52.258673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:50.244428+0000","last_clean_scrub_stamp":"2024-09-10T20:58:50.244428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:35.279483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903782+0000","last_change":"2024-09-10T20:58:47.247788+0000","last_active":"2024-09-10T20:58:53.903782+0000","last_peered":"2024-09-10T20:58:53.903782+0000","last_clean":"2024-09-10T20:58:53.903782+0000","last_became_active":"2024-09-10T20:58:47.247513+0000","last_became_peered":"2024-09-10T20:58:47.247513+0000","last_unstale":"2024-09-10T20:58:53.903782+0000","last_undegraded":"2024-09-10T20:58:53.903782+0000","last_fullsized":"2024-09-10T20:58:53.903782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:59.707834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903324+0000","last_change":"2024-09-10T20:58:47.232038+0000","last_active":"2024-09-10T20:58:53.903324+0000","last_peered":"2024-09-10T20:58:53.903324+0000","last_clean":"2024-09-10T20:58:53.903324+0000","last_became_active":"2024-09-10T20:58:47.231590+0000","last_became_peered":"2024-09-10T20:58:47.231590+0000","last_unstale":"2024-09-10T20:58:53.903324+0000","last_undegraded":"2024-09-10T20:58:53.903324+0000","last_fullsized":"2024-09-10T20:58:53.903324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:26.466044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255659+0000","last_change":"2024-09-10T20:58:47.237625+0000","last_active":"2024-09-10T20:58:52.255659+0000","last_peered":"2024-09-10T20:58:52.255659+0000","last_clean":"2024-09-10T20:58:52.255659+0000","last_became_active":"2024-09-10T20:58:47.237247+0000","last_became_peered":"2024-09-10T20:58:47.237247+0000","last_unstale":"2024-09-10T20:58:52.255659+0000","last_undegraded":"2024-09-10T20:58:52.255659+0000","last_fullsized":"2024-09-10T20:58:52.255659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:21.642999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904061+0000","last_change":"2024-09-10T20:58:47.247495+0000","last_active":"2024-09-10T20:58:53.904061+0000","last_peered":"2024-09-10T20:58:53.904061+0000","last_clean":"2024-09-10T20:58:53.904061+0000","last_became_active":"2024-09-10T20:58:47.247049+0000","last_became_peered":"2024-09-10T20:58:47.247049+0000","last_unstale":"2024-09-10T20:58:53.904061+0000","last_undegraded":"2024-09-10T20:58:53.904061+0000","last_fullsized":"2024-09-10T20:58:53.904061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:18.950297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258717+0000","last_change":"2024-09-10T20:58:47.243173+0000","last_active":"2024-09-10T20:58:52.258717+0000","last_peered":"2024-09-10T20:58:52.258717+0000","last_clean":"2024-09-10T20:58:52.258717+0000","last_became_active":"2024-09-10T20:58:47.242521+0000","last_became_peered":"2024-09-10T20:58:47.242521+0000","last_unstale":"2024-09-10T20:58:52.258717+0000","last_undegraded":"2024-09-10T20:58:52.258717+0000","last_fullsized":"2024-09-10T20:58:52.258717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:54.993134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316600+0000","last_change":"2024-09-10T20:58:47.238462+0000","last_active":"2024-09-10T20:58:53.316600+0000","last_peered":"2024-09-10T20:58:53.316600+0000","last_clean":"2024-09-10T20:58:53.316600+0000","last_became_active":"2024-09-10T20:58:47.238166+0000","last_became_peered":"2024-09-10T20:58:47.238166+0000","last_unstale":"2024-09-10T20:58:53.316600+0000","last_undegraded":"2024-09-10T20:58:53.316600+0000","last_fullsized":"2024-09-10T20:58:53.316600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:51.719957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904758+0000","last_change":"2024-09-10T20:58:47.245403+0000","last_active":"2024-09-10T20:58:53.904758+0000","last_peered":"2024-09-10T20:58:53.904758+0000","last_clean":"2024-09-10T20:58:53.904758+0000","last_became_active":"2024-09-10T20:58:47.245125+0000","last_became_peered":"2024-09-10T20:58:47.245125+0000","last_unstale":"2024-09-10T20:58:53.904758+0000","last_undegraded":"2024-09-10T20:58:53.904758+0000","last_fullsized":"2024-09-10T20:58:53.904758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:29.836234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.259179+0000","last_change":"2024-09-10T20:58:47.242826+0000","last_active":"2024-09-10T20:58:52.259179+0000","last_peered":"2024-09-10T20:58:52.259179+0000","last_clean":"2024-09-10T20:58:52.259179+0000","last_became_active":"2024-09-10T20:58:47.242462+0000","last_became_peered":"2024-09-10T20:58:47.242462+0000","last_unstale":"2024-09-10T20:58:52.259179+0000","last_undegraded":"2024-09-10T20:58:52.259179+0000","last_fullsized":"2024-09-10T20:58:52.259179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:33.221761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255216+0000","last_change":"2024-09-10T20:58:47.236478+0000","last_active":"2024-09-10T20:58:52.255216+0000","last_peered":"2024-09-10T20:58:52.255216+0000","last_clean":"2024-09-10T20:58:52.255216+0000","last_became_active":"2024-09-10T20:58:47.236141+0000","last_became_peered":"2024-09-10T20:58:47.236141+0000","last_unstale":"2024-09-10T20:58:52.255216+0000","last_undegraded":"2024-09-10T20:58:52.255216+0000","last_fullsized":"2024-09-10T20:58:52.255216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:51.719957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258257+0000","last_change":"2024-09-10T20:58:47.233386+0000","last_active":"2024-09-10T20:58:52.258257+0000","last_peered":"2024-09-10T20:58:52.258257+0000","last_clean":"2024-09-10T20:58:52.258257+0000","last_became_active":"2024-09-10T20:58:47.233110+0000","last_became_peered":"2024-09-10T20:58:47.233110+0000","last_unstale":"2024-09-10T20:58:52.258257+0000","last_undegraded":"2024-09-10T20:58:52.258257+0000","last_fullsized":"2024-09-10T20:58:52.258257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:18.950297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255532+0000","last_change":"2024-09-10T20:58:47.236278+0000","last_active":"2024-09-10T20:58:52.255532+0000","last_peered":"2024-09-10T20:58:52.255532+0000","last_clean":"2024-09-10T20:58:52.255532+0000","last_became_active":"2024-09-10T20:58:47.236005+0000","last_became_peered":"2024-09-10T20:58:47.236005+0000","last_unstale":"2024-09-10T20:58:52.255532+0000","last_undegraded":"2024-09-10T20:58:52.255532+0000","last_fullsized":"2024-09-10T20:58:52.255532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:34:54.893698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903878+0000","last_change":"2024-09-10T20:58:47.247778+0000","last_active":"2024-09-10T20:58:53.903878+0000","last_peered":"2024-09-10T20:58:53.903878+0000","last_clean":"2024-09-10T20:58:53.903878+0000","last_became_active":"2024-09-10T20:58:47.247103+0000","last_became_peered":"2024-09-10T20:58:47.247103+0000","last_unstale":"2024-09-10T20:58:53.903878+0000","last_undegraded":"2024-09-10T20:58:53.903878+0000","last_fullsized":"2024-09-10T20:58:53.903878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:42.546277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904334+0000","last_change":"2024-09-10T20:58:47.230471+0000","last_active":"2024-09-10T20:58:53.904334+0000","last_peered":"2024-09-10T20:58:53.904334+0000","last_clean":"2024-09-10T20:58:53.904334+0000","last_became_active":"2024-09-10T20:58:47.230174+0000","last_became_peered":"2024-09-10T20:58:47.230174+0000","last_unstale":"2024-09-10T20:58:53.904334+0000","last_undegraded":"2024-09-10T20:58:53.904334+0000","last_fullsized":"2024-09-10T20:58:53.904334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:54.301892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905155+0000","last_change":"2024-09-10T20:58:47.245889+0000","last_active":"2024-09-10T20:58:53.905155+0000","last_peered":"2024-09-10T20:58:53.905155+0000","last_clean":"2024-09-10T20:58:53.905155+0000","last_became_active":"2024-09-10T20:58:47.245547+0000","last_became_peered":"2024-09-10T20:58:47.245547+0000","last_unstale":"2024-09-10T20:58:53.905155+0000","last_undegraded":"2024-09-10T20:58:53.905155+0000","last_fullsized":"2024-09-10T20:58:53.905155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:11.073447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903659+0000","last_change":"2024-09-10T20:58:47.234594+0000","last_active":"2024-09-10T20:58:53.903659+0000","last_peered":"2024-09-10T20:58:53.903659+0000","last_clean":"2024-09-10T20:58:53.903659+0000","last_became_active":"2024-09-10T20:58:47.234275+0000","last_became_peered":"2024-09-10T20:58:47.234275+0000","last_unstale":"2024-09-10T20:58:53.903659+0000","last_undegraded":"2024-09-10T20:58:53.903659+0000","last_fullsized":"2024-09-10T20:58:53.903659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:25.490881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.256889+0000","last_change":"2024-09-10T20:58:47.236653+0000","last_active":"2024-09-10T20:58:52.256889+0000","last_peered":"2024-09-10T20:58:52.256889+0000","last_clean":"2024-09-10T20:58:52.256889+0000","last_became_active":"2024-09-10T20:58:47.236361+0000","last_became_peered":"2024-09-10T20:58:47.236361+0000","last_unstale":"2024-09-10T20:58:52.256889+0000","last_undegraded":"2024-09-10T20:58:52.256889+0000","last_fullsized":"2024-09-10T20:58:52.256889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:50.078953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255103+0000","last_change":"2024-09-10T20:58:47.247779+0000","last_active":"2024-09-10T20:58:52.255103+0000","last_peered":"2024-09-10T20:58:52.255103+0000","last_clean":"2024-09-10T20:58:52.255103+0000","last_became_active":"2024-09-10T20:58:47.246921+0000","last_became_peered":"2024-09-10T20:58:47.246921+0000","last_unstale":"2024-09-10T20:58:52.255103+0000","last_undegraded":"2024-09-10T20:58:52.255103+0000","last_fullsized":"2024-09-10T20:58:52.255103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:07.109711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316370+0000","last_change":"2024-09-10T20:58:47.233169+0000","last_active":"2024-09-10T20:58:53.316370+0000","last_peered":"2024-09-10T20:58:53.316370+0000","last_clean":"2024-09-10T20:58:53.316370+0000","last_became_active":"2024-09-10T20:58:47.232869+0000","last_became_peered":"2024-09-10T20:58:47.232869+0000","last_unstale":"2024-09-10T20:58:53.316370+0000","last_undegraded":"2024-09-10T20:58:53.316370+0000","last_fullsized":"2024-09-10T20:58:53.316370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:37.216505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.903723+0000","last_change":"2024-09-10T20:58:47.241778+0000","last_active":"2024-09-10T20:58:53.903723+0000","last_peered":"2024-09-10T20:58:53.903723+0000","last_clean":"2024-09-10T20:58:53.903723+0000","last_became_active":"2024-09-10T20:58:47.241164+0000","last_became_peered":"2024-09-10T20:58:47.241164+0000","last_unstale":"2024-09-10T20:58:53.903723+0000","last_undegraded":"2024-09-10T20:58:53.903723+0000","last_fullsized":"2024-09-10T20:58:53.903723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:59.707834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258272+0000","last_change":"2024-09-10T20:58:47.234831+0000","last_active":"2024-09-10T20:58:52.258272+0000","last_peered":"2024-09-10T20:58:52.258272+0000","last_clean":"2024-09-10T20:58:52.258272+0000","last_became_active":"2024-09-10T20:58:47.234566+0000","last_became_peered":"2024-09-10T20:58:47.234566+0000","last_unstale":"2024-09-10T20:58:52.258272+0000","last_undegraded":"2024-09-10T20:58:52.258272+0000","last_fullsized":"2024-09-10T20:58:52.258272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:29.836234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.257006+0000","last_change":"2024-09-10T20:58:47.231297+0000","last_active":"2024-09-10T20:58:52.257006+0000","last_peered":"2024-09-10T20:58:52.257006+0000","last_clean":"2024-09-10T20:58:52.257006+0000","last_became_active":"2024-09-10T20:58:47.230943+0000","last_became_peered":"2024-09-10T20:58:47.230943+0000","last_unstale":"2024-09-10T20:58:52.257006+0000","last_undegraded":"2024-09-10T20:58:52.257006+0000","last_fullsized":"2024-09-10T20:58:52.257006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:29.113846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904210+0000","last_change":"2024-09-10T20:58:47.232404+0000","last_active":"2024-09-10T20:58:53.904210+0000","last_peered":"2024-09-10T20:58:53.904210+0000","last_clean":"2024-09-10T20:58:53.904210+0000","last_became_active":"2024-09-10T20:58:47.232127+0000","last_became_peered":"2024-09-10T20:58:47.232127+0000","last_unstale":"2024-09-10T20:58:53.904210+0000","last_undegraded":"2024-09-10T20:58:53.904210+0000","last_fullsized":"2024-09-10T20:58:53.904210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:39.726666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904756+0000","last_change":"2024-09-10T20:58:47.231816+0000","last_active":"2024-09-10T20:58:53.904756+0000","last_peered":"2024-09-10T20:58:53.904756+0000","last_clean":"2024-09-10T20:58:53.904756+0000","last_became_active":"2024-09-10T20:58:47.231545+0000","last_became_peered":"2024-09-10T20:58:47.231545+0000","last_unstale":"2024-09-10T20:58:53.904756+0000","last_undegraded":"2024-09-10T20:58:53.904756+0000","last_fullsized":"2024-09-10T20:58:53.904756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:54.993134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.905112+0000","last_change":"2024-09-10T20:58:47.246656+0000","last_active":"2024-09-10T20:58:53.905112+0000","last_peered":"2024-09-10T20:58:53.905112+0000","last_clean":"2024-09-10T20:58:53.905112+0000","last_became_active":"2024-09-10T20:58:47.245647+0000","last_became_peered":"2024-09-10T20:58:47.245647+0000","last_unstale":"2024-09-10T20:58:53.905112+0000","last_undegraded":"2024-09-10T20:58:53.905112+0000","last_fullsized":"2024-09-10T20:58:53.905112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:35.158790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.258006+0000","last_change":"2024-09-10T20:58:47.235102+0000","last_active":"2024-09-10T20:58:52.258006+0000","last_peered":"2024-09-10T20:58:52.258006+0000","last_clean":"2024-09-10T20:58:52.258006+0000","last_became_active":"2024-09-10T20:58:47.233909+0000","last_became_peered":"2024-09-10T20:58:47.233909+0000","last_unstale":"2024-09-10T20:58:52.258006+0000","last_undegraded":"2024-09-10T20:58:52.258006+0000","last_fullsized":"2024-09-10T20:58:52.258006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:59.707834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904152+0000","last_change":"2024-09-10T20:58:47.242793+0000","last_active":"2024-09-10T20:58:53.904152+0000","last_peered":"2024-09-10T20:58:53.904152+0000","last_clean":"2024-09-10T20:58:53.904152+0000","last_became_active":"2024-09-10T20:58:47.242495+0000","last_became_peered":"2024-09-10T20:58:47.242495+0000","last_unstale":"2024-09-10T20:58:53.904152+0000","last_undegraded":"2024-09-10T20:58:53.904152+0000","last_fullsized":"2024-09-10T20:58:53.904152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:21.457962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255428+0000","last_change":"2024-09-10T20:58:47.244668+0000","last_active":"2024-09-10T20:58:52.255428+0000","last_peered":"2024-09-10T20:58:52.255428+0000","last_clean":"2024-09-10T20:58:52.255428+0000","last_became_active":"2024-09-10T20:58:47.244110+0000","last_became_peered":"2024-09-10T20:58:47.244110+0000","last_unstale":"2024-09-10T20:58:52.255428+0000","last_undegraded":"2024-09-10T20:58:52.255428+0000","last_fullsized":"2024-09-10T20:58:52.255428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:41.633550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904570+0000","last_change":"2024-09-10T20:58:47.241409+0000","last_active":"2024-09-10T20:58:53.904570+0000","last_peered":"2024-09-10T20:58:53.904570+0000","last_clean":"2024-09-10T20:58:53.904570+0000","last_became_active":"2024-09-10T20:58:47.241104+0000","last_became_peered":"2024-09-10T20:58:47.241104+0000","last_unstale":"2024-09-10T20:58:53.904570+0000","last_undegraded":"2024-09-10T20:58:53.904570+0000","last_fullsized":"2024-09-10T20:58:53.904570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:08.330706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:52.255561+0000","last_change":"2024-09-10T20:58:47.246735+0000","last_active":"2024-09-10T20:58:52.255561+0000","last_peered":"2024-09-10T20:58:52.255561+0000","last_clean":"2024-09-10T20:58:52.255561+0000","last_became_active":"2024-09-10T20:58:47.246491+0000","last_became_peered":"2024-09-10T20:58:47.246491+0000","last_unstale":"2024-09-10T20:58:52.255561+0000","last_undegraded":"2024-09-10T20:58:52.255561+0000","last_fullsized":"2024-09-10T20:58:52.255561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:16.591593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904106+0000","last_change":"2024-09-10T20:58:47.246507+0000","last_active":"2024-09-10T20:58:53.904106+0000","last_peered":"2024-09-10T20:58:53.904106+0000","last_clean":"2024-09-10T20:58:53.904106+0000","last_became_active":"2024-09-10T20:58:47.245963+0000","last_became_peered":"2024-09-10T20:58:47.245963+0000","last_unstale":"2024-09-10T20:58:53.904106+0000","last_undegraded":"2024-09-10T20:58:53.904106+0000","last_fullsized":"2024-09-10T20:58:53.904106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:04.498018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.316869+0000","last_change":"2024-09-10T20:58:47.242014+0000","last_active":"2024-09-10T20:58:53.316869+0000","last_peered":"2024-09-10T20:58:53.316869+0000","last_clean":"2024-09-10T20:58:53.316869+0000","last_became_active":"2024-09-10T20:58:47.241727+0000","last_became_peered":"2024-09-10T20:58:47.241727+0000","last_unstale":"2024-09-10T20:58:53.316869+0000","last_undegraded":"2024-09-10T20:58:53.316869+0000","last_fullsized":"2024-09-10T20:58:53.316869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:13.462491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T20:58:53.904966+0000","last_change":"2024-09-10T20:58:47.246360+0000","last_active":"2024-09-10T20:58:53.904966+0000","last_peered":"2024-09-10T20:58:53.904966+0000","last_clean":"2024-09-10T20:58:53.904966+0000","last_became_active":"2024-09-10T20:58:47.245363+0000","last_became_peered":"2024-09-10T20:58:47.245363+0000","last_unstale":"2024-09-10T20:58:53.904966+0000","last_undegraded":"2024-09-10T20:58:53.904966+0000","last_fullsized":"2024-09-10T20:58:53.904966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:46.203356+0000","last_clean_scrub_stamp":"2024-09-10T20:58:46.203356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:59.710882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":693653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":102829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":102863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":693709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":262144,"data_stored":102793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:59:12.187 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T20:59:12.187 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T20:59:12.187 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T20:59:12.188 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T20:59:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.318+0000 7fd4922b8640 1 -- 172.21.15.98:0/3629538530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd48c06cb70 msgr2=0x7fd48c0cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.318+0000 7fd4922b8640 1 --2- 172.21.15.98:0/3629538530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd48c06cb70 0x7fd48c0cb080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd484005fd0 tx=0x7fd48402a8e0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.318+0000 7fd4922b8640 1 -- 172.21.15.98:0/3629538530 shutdown_connections 2024-09-10T20:59:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.318+0000 7fd4922b8640 1 --2- 172.21.15.98:0/3629538530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd48c06e0a0 0x7fd48c0b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.318+0000 7fd4922b8640 1 --2- 172.21.15.98:0/3629538530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd48c06cb70 0x7fd48c0cb080 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.318+0000 7fd4922b8640 1 -- 172.21.15.98:0/3629538530 >> 172.21.15.98:0/3629538530 conn(0x7fd48c074150 msgr2=0x7fd48c074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.318+0000 7fd4922b8640 1 -- 172.21.15.98:0/3629538530 shutdown_connections 2024-09-10T20:59:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.318+0000 7fd4922b8640 1 -- 172.21.15.98:0/3629538530 wait complete. 2024-09-10T20:59:12.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.320+0000 7fd4922b8640 1 Processor -- start 2024-09-10T20:59:12.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.320+0000 7fd4922b8640 1 -- start start 2024-09-10T20:59:12.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd4922b8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd48c06cb70 0x7fd48c0992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:12.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd4922b8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd48c06dcc0 0x7fd48c1f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:12.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd4922b8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd48c06e0a0 0x7fd48c1f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:12.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd4922b8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c0b5010 con 0x7fd48c06dcc0 2024-09-10T20:59:12.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd4922b8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c069490 con 0x7fd48c06cb70 2024-09-10T20:59:12.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd4922b8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd48c0b5190 con 0x7fd48c06e0a0 2024-09-10T20:59:12.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd48b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd48c06dcc0 0x7fd48c1f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:12.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd48b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd48c06dcc0 0x7fd48c1f4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55476/0 (socket says 172.21.15.98:55476) 2024-09-10T20:59:12.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd48b7fe640 1 -- 172.21.15.98:0/671461029 learned_addr learned my addr 172.21.15.98:0/671461029 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:12.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd48bfff640 1 --2- 172.21.15.98:0/671461029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd48c06cb70 0x7fd48c0992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:12.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd490855640 1 --2- 172.21.15.98:0/671461029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd48c06e0a0 0x7fd48c1f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:12.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd48b7fe640 1 -- 172.21.15.98:0/671461029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd48c06e0a0 msgr2=0x7fd48c1f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:12.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd48b7fe640 1 --2- 172.21.15.98:0/671461029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd48c06e0a0 0x7fd48c1f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd48b7fe640 1 -- 172.21.15.98:0/671461029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd48c06cb70 msgr2=0x7fd48c0992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:12.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd48b7fe640 1 --2- 172.21.15.98:0/671461029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd48c06cb70 0x7fd48c0992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd48b7fe640 1 -- 172.21.15.98:0/671461029 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c15de30 con 0x7fd48c06dcc0 2024-09-10T20:59:12.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.321+0000 7fd48b7fe640 1 --2- 172.21.15.98:0/671461029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd48c06dcc0 0x7fd48c1f4930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c970 tx=0x7fd48000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:12.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.324+0000 7fd488ff9640 1 -- 172.21.15.98:0/671461029 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480004220 con 0x7fd48c06dcc0 2024-09-10T20:59:12.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.325+0000 7fd488ff9640 1 -- 172.21.15.98:0/671461029 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4800043e0 con 0x7fd48c06dcc0 2024-09-10T20:59:12.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.325+0000 7fd4922b8640 1 -- 172.21.15.98:0/671461029 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c15e110 con 0x7fd48c06dcc0 2024-09-10T20:59:12.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.325+0000 7fd488ff9640 1 -- 172.21.15.98:0/671461029 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480007c10 con 0x7fd48c06dcc0 2024-09-10T20:59:12.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.325+0000 7fd4922b8640 1 -- 172.21.15.98:0/671461029 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c15e6d0 con 0x7fd48c06dcc0 2024-09-10T20:59:12.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.325+0000 7fd4922b8640 1 -- 172.21.15.98:0/671461029 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c0ad250 con 0x7fd48c06dcc0 2024-09-10T20:59:12.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.326+0000 7fd488ff9640 1 -- 172.21.15.98:0/671461029 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd480020060 con 0x7fd48c06dcc0 2024-09-10T20:59:12.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.327+0000 7fd488ff9640 1 --2- 172.21.15.98:0/671461029 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd478077f70 0x7fd47807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:12.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.327+0000 7fd488ff9640 1 -- 172.21.15.98:0/671461029 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd480099dd0 con 0x7fd48c06dcc0 2024-09-10T20:59:12.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.327+0000 7fd48bfff640 1 --2- 172.21.15.98:0/671461029 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd478077f70 0x7fd47807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:12.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.327+0000 7fd48bfff640 1 --2- 172.21.15.98:0/671461029 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd478077f70 0x7fd47807a430 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd48c152b50 tx=0x7fd47c007e30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:12.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.331+0000 7fd488ff9640 1 -- 172.21.15.98:0/671461029 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd48c0ad250 con 0x7fd48c06dcc0 2024-09-10T20:59:12.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.543+0000 7fd4922b8640 1 -- 172.21.15.98:0/671461029 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd48c0b1720 con 0x7fd48c06dcc0 2024-09-10T20:59:12.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.543+0000 7fd488ff9640 1 -- 172.21.15.98:0/671461029 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd480014030 con 0x7fd48c06dcc0 2024-09-10T20:59:12.547 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:12.547 INFO:teuthology.orchestra.run.smithi098.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T20:59:12.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.545+0000 7fd4922b8640 1 -- 172.21.15.98:0/671461029 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd478077f70 msgr2=0x7fd47807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:12.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.545+0000 7fd4922b8640 1 --2- 172.21.15.98:0/671461029 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd478077f70 0x7fd47807a430 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd48c152b50 tx=0x7fd47c007e30 comp rx=0 tx=0).stop 2024-09-10T20:59:12.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.545+0000 7fd4922b8640 1 -- 172.21.15.98:0/671461029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd48c06dcc0 msgr2=0x7fd48c1f4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:12.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.545+0000 7fd4922b8640 1 --2- 172.21.15.98:0/671461029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd48c06dcc0 0x7fd48c1f4930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c970 tx=0x7fd48000ce30 comp rx=0 tx=0).stop 2024-09-10T20:59:12.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.545+0000 7fd4922b8640 1 -- 172.21.15.98:0/671461029 shutdown_connections 2024-09-10T20:59:12.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.545+0000 7fd4922b8640 1 --2- 172.21.15.98:0/671461029 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd478077f70 0x7fd47807a430 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.545+0000 7fd4922b8640 1 --2- 172.21.15.98:0/671461029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd48c06e0a0 0x7fd48c1f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.545+0000 7fd4922b8640 1 --2- 172.21.15.98:0/671461029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd48c06dcc0 0x7fd48c1f4930 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.545+0000 7fd4922b8640 1 --2- 172.21.15.98:0/671461029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd48c06cb70 0x7fd48c0992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.545+0000 7fd4922b8640 1 -- 172.21.15.98:0/671461029 >> 172.21.15.98:0/671461029 conn(0x7fd48c074150 msgr2=0x7fd48c0ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:12.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.545+0000 7fd4922b8640 1 -- 172.21.15.98:0/671461029 shutdown_connections 2024-09-10T20:59:12.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.545+0000 7fd4922b8640 1 -- 172.21.15.98:0/671461029 wait complete. 2024-09-10T20:59:12.560 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T20:59:12.560 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:59:12.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.657+0000 7fa34fd4b640 1 -- 172.21.15.98:0/2071027107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3480d4af0 msgr2=0x7fa3480d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:12.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.657+0000 7fa34fd4b640 1 --2- 172.21.15.98:0/2071027107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3480d4af0 0x7fa3480d2a20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa34806d2a0 tx=0x7fa34002aa50 comp rx=0 tx=0).stop 2024-09-10T20:59:12.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.657+0000 7fa34fd4b640 1 -- 172.21.15.98:0/2071027107 shutdown_connections 2024-09-10T20:59:12.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.657+0000 7fa34fd4b640 1 --2- 172.21.15.98:0/2071027107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3480d4af0 0x7fa3480d2a20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.657+0000 7fa34fd4b640 1 --2- 172.21.15.98:0/2071027107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3480b7300 0x7fa3480d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.657+0000 7fa34fd4b640 1 --2- 172.21.15.98:0/2071027107 >> v2:172.21.15.190:3300/0 conn(0x7fa34815e060 0x7fa3480b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.657+0000 7fa34fd4b640 1 -- 172.21.15.98:0/2071027107 >> 172.21.15.98:0/2071027107 conn(0x7fa3480d3f70 msgr2=0x7fa3480d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:12.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.657+0000 7fa34fd4b640 1 -- 172.21.15.98:0/2071027107 shutdown_connections 2024-09-10T20:59:12.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.657+0000 7fa34fd4b640 1 -- 172.21.15.98:0/2071027107 wait complete. 2024-09-10T20:59:12.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.658+0000 7fa34fd4b640 1 Processor -- start 2024-09-10T20:59:12.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.658+0000 7fa34fd4b640 1 -- start start 2024-09-10T20:59:12.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34fd4b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3480d4af0 0x7fa348160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:12.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34fd4b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa34815e060 0x7fa3480cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:12.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34fd4b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3480ccd40 0x7fa3481f8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:12.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34fd4b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa348161e60 con 0x7fa3480ccd40 2024-09-10T20:59:12.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34fd4b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa348161ce0 con 0x7fa3480d4af0 2024-09-10T20:59:12.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34fd4b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa348161fe0 con 0x7fa34815e060 2024-09-10T20:59:12.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34e2e8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3480ccd40 0x7fa3481f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:12.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34e2e8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3480ccd40 0x7fa3481f8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55488/0 (socket says 172.21.15.98:55488) 2024-09-10T20:59:12.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34e2e8640 1 -- 172.21.15.98:0/413533600 learned_addr learned my addr 172.21.15.98:0/413533600 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:12.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34dae7640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3480d4af0 0x7fa348160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:12.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34d2e6640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa34815e060 0x7fa3480cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:12.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34e2e8640 1 -- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa34815e060 msgr2=0x7fa3480cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:12.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34e2e8640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa34815e060 0x7fa3480cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34e2e8640 1 -- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3480d4af0 msgr2=0x7fa348160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:12.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34e2e8640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3480d4af0 0x7fa348160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34e2e8640 1 -- 172.21.15.98:0/413533600 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3481f4970 con 0x7fa3480ccd40 2024-09-10T20:59:12.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34d2e6640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa34815e060 0x7fa3480cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:12.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34e2e8640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3480ccd40 0x7fa3481f8080 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa340006010 tx=0x7fa340039a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:12.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa33e7fc640 1 -- 172.21.15.98:0/413533600 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340033040 con 0x7fa3480ccd40 2024-09-10T20:59:12.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa33e7fc640 1 -- 172.21.15.98:0/413533600 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa340003970 con 0x7fa3480ccd40 2024-09-10T20:59:12.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa33e7fc640 1 -- 172.21.15.98:0/413533600 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340003dc0 con 0x7fa3480ccd40 2024-09-10T20:59:12.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.659+0000 7fa34fd4b640 1 -- 172.21.15.98:0/413533600 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3481f4c50 con 0x7fa3480ccd40 2024-09-10T20:59:12.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.660+0000 7fa34fd4b640 1 -- 172.21.15.98:0/413533600 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3481f5710 con 0x7fa3480ccd40 2024-09-10T20:59:12.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.660+0000 7fa34fd4b640 1 -- 172.21.15.98:0/413533600 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3480b6dc0 con 0x7fa3480ccd40 2024-09-10T20:59:12.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.661+0000 7fa33e7fc640 1 -- 172.21.15.98:0/413533600 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa34002f6e0 con 0x7fa3480ccd40 2024-09-10T20:59:12.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.661+0000 7fa33e7fc640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa32c077fa0 0x7fa32c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:12.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.662+0000 7fa33e7fc640 1 -- 172.21.15.98:0/413533600 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa3400bb790 con 0x7fa3480ccd40 2024-09-10T20:59:12.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.662+0000 7fa34dae7640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa32c077fa0 0x7fa32c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:12.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.662+0000 7fa34dae7640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa32c077fa0 0x7fa32c07a460 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa3480b7f70 tx=0x7fa34400b2e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:12.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.666+0000 7fa33e7fc640 1 -- 172.21.15.98:0/413533600 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa340083f50 con 0x7fa3480ccd40 2024-09-10T20:59:12.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.877+0000 7fa34fd4b640 1 -- 172.21.15.98:0/413533600 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3481f4e20 con 0x7fa3480ccd40 2024-09-10T20:59:12.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.878+0000 7fa33e7fc640 1 -- 172.21.15.98:0/413533600 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7fa3400098b0 con 0x7fa3480ccd40 2024-09-10T20:59:12.882 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:12.882 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"btime":"2024-09-10T20:59:06:294252+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4277,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/1954552281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1954552281},{"type":"v1","addr":"172.21.15.190:6836","nonce":1954552281}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/8997965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":8997965},{"type":"v1","addr":"172.21.15.190:6837","nonce":8997965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1984550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1984550306},{"type":"v1","addr":"172.21.15.98:6833","nonce":1984550306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:58:46.560479+0000","modified":"2024-09-10T20:59:05.635381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3141703317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3141703317},{"type":"v1","addr":"172.21.15.98:6835","nonce":3141703317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T20:59:12.883 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2024-09-10T20:59:12.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.880+0000 7fa34fd4b640 1 -- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa32c077fa0 msgr2=0x7fa32c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:12.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.880+0000 7fa34fd4b640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa32c077fa0 0x7fa32c07a460 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa3480b7f70 tx=0x7fa34400b2e0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.880+0000 7fa34fd4b640 1 -- 172.21.15.98:0/413533600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3480ccd40 msgr2=0x7fa3481f8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:12.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.880+0000 7fa34fd4b640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3480ccd40 0x7fa3481f8080 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa340006010 tx=0x7fa340039a80 comp rx=0 tx=0).stop 2024-09-10T20:59:12.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.880+0000 7fa34fd4b640 1 -- 172.21.15.98:0/413533600 shutdown_connections 2024-09-10T20:59:12.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.880+0000 7fa34fd4b640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa32c077fa0 0x7fa32c07a460 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.880+0000 7fa34fd4b640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3480ccd40 0x7fa3481f8080 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.880+0000 7fa34fd4b640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa34815e060 0x7fa3480cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.880+0000 7fa34fd4b640 1 --2- 172.21.15.98:0/413533600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3480d4af0 0x7fa348160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.880+0000 7fa34fd4b640 1 -- 172.21.15.98:0/413533600 >> 172.21.15.98:0/413533600 conn(0x7fa3480d3f70 msgr2=0x7fa3480b94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:12.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.880+0000 7fa34fd4b640 1 -- 172.21.15.98:0/413533600 shutdown_connections 2024-09-10T20:59:12.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.880+0000 7fa34fd4b640 1 -- 172.21.15.98:0/413533600 wait complete. 2024-09-10T20:59:12.895 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:59:12.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.992+0000 7f6d1ed53640 1 -- 172.21.15.98:0/3580209065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d18070470 msgr2=0x7f6d180b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:12.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.992+0000 7f6d1ed53640 1 --2- 172.21.15.98:0/3580209065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d18070470 0x7f6d180b2f80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6d10005fd0 tx=0x7f6d1002aa90 comp rx=0 tx=0).stop 2024-09-10T20:59:12.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.992+0000 7f6d1ed53640 1 -- 172.21.15.98:0/3580209065 shutdown_connections 2024-09-10T20:59:12.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.992+0000 7f6d1ed53640 1 --2- 172.21.15.98:0/3580209065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d18070470 0x7f6d180b2f80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.992+0000 7f6d1ed53640 1 --2- 172.21.15.98:0/3580209065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d18157fe0 0x7f6d180733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:12.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.992+0000 7f6d1ed53640 1 -- 172.21.15.98:0/3580209065 >> 172.21.15.98:0/3580209065 conn(0x7f6d180b1c80 msgr2=0x7f6d180b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:12.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.992+0000 7f6d1ed53640 1 -- 172.21.15.98:0/3580209065 shutdown_connections 2024-09-10T20:59:12.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.992+0000 7f6d1ed53640 1 -- 172.21.15.98:0/3580209065 wait complete. 2024-09-10T20:59:12.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.993+0000 7f6d1ed53640 1 Processor -- start 2024-09-10T20:59:12.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.993+0000 7f6d1ed53640 1 -- start start 2024-09-10T20:59:12.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1ed53640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d18070470 0x7f6d181572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:12.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1ed53640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d18157fe0 0x7f6d1814b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:12.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1ed53640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d1815b2d0 0x7f6d1814cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:13.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1ed53640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d180b38d0 con 0x7f6d1815b2d0 2024-09-10T20:59:13.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1ed53640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6d180b3750 con 0x7f6d18070470 2024-09-10T20:59:13.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1ed53640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6d180b3a50 con 0x7f6d18157fe0 2024-09-10T20:59:13.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1d2f0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d1815b2d0 0x7f6d1814cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:13.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1d2f0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d1815b2d0 0x7f6d1814cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55500/0 (socket says 172.21.15.98:55500) 2024-09-10T20:59:13.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1d2f0640 1 -- 172.21.15.98:0/37881165 learned_addr learned my addr 172.21.15.98:0/37881165 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:13.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1caef640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d18070470 0x7f6d181572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:13.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d0ffff640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d18157fe0 0x7f6d1814b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:13.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1d2f0640 1 -- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d18157fe0 msgr2=0x7f6d1814b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:13.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1d2f0640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d18157fe0 0x7f6d1814b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1d2f0640 1 -- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d18070470 msgr2=0x7f6d181572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:13.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1d2f0640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d18070470 0x7f6d181572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1d2f0640 1 -- 172.21.15.98:0/37881165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d181577e0 con 0x7f6d1815b2d0 2024-09-10T20:59:13.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1caef640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d18070470 0x7f6d181572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:13.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d0ffff640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d18157fe0 0x7f6d1814b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:13.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1d2f0640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d1815b2d0 0x7f6d1814cfe0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6d10006010 tx=0x7f6d10003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:13.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d0d7fa640 1 -- 172.21.15.98:0/37881165 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10046070 con 0x7f6d1815b2d0 2024-09-10T20:59:13.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d0d7fa640 1 -- 172.21.15.98:0/37881165 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d10003e50 con 0x7f6d1815b2d0 2024-09-10T20:59:13.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.994+0000 7f6d1ed53640 1 -- 172.21.15.98:0/37881165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d181f5f80 con 0x7f6d1815b2d0 2024-09-10T20:59:13.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.995+0000 7f6d0d7fa640 1 -- 172.21.15.98:0/37881165 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d100044b0 con 0x7f6d1815b2d0 2024-09-10T20:59:13.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.995+0000 7f6d1ed53640 1 -- 172.21.15.98:0/37881165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d181f6540 con 0x7f6d1815b2d0 2024-09-10T20:59:13.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.996+0000 7f6d1ed53640 1 -- 172.21.15.98:0/37881165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d180b3f00 con 0x7f6d1815b2d0 2024-09-10T20:59:13.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.996+0000 7f6d0d7fa640 1 -- 172.21.15.98:0/37881165 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6d10004650 con 0x7f6d1815b2d0 2024-09-10T20:59:13.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.997+0000 7f6d0d7fa640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6d04077fa0 0x7f6d0407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:13.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.997+0000 7f6d0d7fa640 1 -- 172.21.15.98:0/37881165 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6d100bb1a0 con 0x7f6d1815b2d0 2024-09-10T20:59:13.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.997+0000 7f6d1caef640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6d04077fa0 0x7f6d0407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:13.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:12.997+0000 7f6d1caef640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6d04077fa0 0x7f6d0407a460 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6d140058c0 tx=0x7f6d14005850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:13.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.001+0000 7f6d0d7fa640 1 -- 172.21.15.98:0/37881165 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6d10032030 con 0x7f6d1815b2d0 2024-09-10T20:59:13.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.211+0000 7f6d1ed53640 1 -- 172.21.15.98:0/37881165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d180b0350 con 0x7f6d1815b2d0 2024-09-10T20:59:13.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.211+0000 7f6d0d7fa640 1 -- 172.21.15.98:0/37881165 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7f6d100839f0 con 0x7f6d1815b2d0 2024-09-10T20:59:13.215 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:13.216 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"btime":"2024-09-10T20:59:06:294252+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4277,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/1954552281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1954552281},{"type":"v1","addr":"172.21.15.190:6836","nonce":1954552281}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/8997965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":8997965},{"type":"v1","addr":"172.21.15.190:6837","nonce":8997965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1984550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1984550306},{"type":"v1","addr":"172.21.15.98:6833","nonce":1984550306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:58:46.560479+0000","modified":"2024-09-10T20:59:05.635381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3141703317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3141703317},{"type":"v1","addr":"172.21.15.98:6835","nonce":3141703317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T20:59:13.216 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2024-09-10T20:59:13.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.213+0000 7f6d1ed53640 1 -- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6d04077fa0 msgr2=0x7f6d0407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:13.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.213+0000 7f6d1ed53640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6d04077fa0 0x7f6d0407a460 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6d140058c0 tx=0x7f6d14005850 comp rx=0 tx=0).stop 2024-09-10T20:59:13.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.213+0000 7f6d1ed53640 1 -- 172.21.15.98:0/37881165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d1815b2d0 msgr2=0x7f6d1814cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:13.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.213+0000 7f6d1ed53640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d1815b2d0 0x7f6d1814cfe0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6d10006010 tx=0x7f6d10003ce0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.213+0000 7f6d1ed53640 1 -- 172.21.15.98:0/37881165 shutdown_connections 2024-09-10T20:59:13.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.213+0000 7f6d1ed53640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6d04077fa0 0x7f6d0407a460 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.213+0000 7f6d1ed53640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d1815b2d0 0x7f6d1814cfe0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.213+0000 7f6d1ed53640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d18157fe0 0x7f6d1814b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.213+0000 7f6d1ed53640 1 --2- 172.21.15.98:0/37881165 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d18070470 0x7f6d181572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.213+0000 7f6d1ed53640 1 -- 172.21.15.98:0/37881165 >> 172.21.15.98:0/37881165 conn(0x7f6d180b1c80 msgr2=0x7f6d180b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:13.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.214+0000 7f6d1ed53640 1 -- 172.21.15.98:0/37881165 shutdown_connections 2024-09-10T20:59:13.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.214+0000 7f6d1ed53640 1 -- 172.21.15.98:0/37881165 wait complete. 2024-09-10T20:59:13.229 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:59:13.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.325+0000 7f4fcc802640 1 -- 172.21.15.98:0/3500056202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc8073ae0 msgr2=0x7f4fc80b3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:13.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.325+0000 7f4fcc802640 1 --2- 172.21.15.98:0/3500056202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc8073ae0 0x7f4fc80b3920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009c10 tx=0x7f4fc002a940 comp rx=0 tx=0).stop 2024-09-10T20:59:13.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.325+0000 7f4fcc802640 1 -- 172.21.15.98:0/3500056202 shutdown_connections 2024-09-10T20:59:13.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.325+0000 7f4fcc802640 1 --2- 172.21.15.98:0/3500056202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc8073ae0 0x7f4fc80b3920 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.325+0000 7f4fcc802640 1 -- 172.21.15.98:0/3500056202 >> 172.21.15.98:0/3500056202 conn(0x7f4fc80d0970 msgr2=0x7f4fc80d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:13.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.325+0000 7f4fcc802640 1 -- 172.21.15.98:0/3500056202 shutdown_connections 2024-09-10T20:59:13.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.325+0000 7f4fcc802640 1 -- 172.21.15.98:0/3500056202 wait complete. 2024-09-10T20:59:13.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.327+0000 7f4fcc802640 1 Processor -- start 2024-09-10T20:59:13.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.327+0000 7f4fcc802640 1 -- start start 2024-09-10T20:59:13.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.327+0000 7f4fcc802640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc8073ae0 0x7f4fc81f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:13.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.327+0000 7f4fcc802640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc8154bf0 0x7f4fc81598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:13.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.327+0000 7f4fcc802640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc815b930 0x7f4fc8159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:13.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.327+0000 7f4fcc802640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc80b41f0 con 0x7f4fc815b930 2024-09-10T20:59:13.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.327+0000 7f4fcc802640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc80b4070 con 0x7f4fc8154bf0 2024-09-10T20:59:13.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.327+0000 7f4fcc802640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4fc80b4370 con 0x7f4fc8073ae0 2024-09-10T20:59:13.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.327+0000 7f4fc759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc815b930 0x7f4fc8159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:13.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.327+0000 7f4fc759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc815b930 0x7f4fc8159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55502/0 (socket says 172.21.15.98:55502) 2024-09-10T20:59:13.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.327+0000 7f4fc759e640 1 -- 172.21.15.98:0/2556607235 learned_addr learned my addr 172.21.15.98:0/2556607235 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:13.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.328+0000 7f4fbffff640 1 --2- 172.21.15.98:0/2556607235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc8154bf0 0x7f4fc81598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:13.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.328+0000 7f4fc6d9d640 1 --2- 172.21.15.98:0/2556607235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc8073ae0 0x7f4fc81f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:13.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.328+0000 7f4fc759e640 1 -- 172.21.15.98:0/2556607235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc8073ae0 msgr2=0x7f4fc81f4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:13.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.328+0000 7f4fc759e640 1 --2- 172.21.15.98:0/2556607235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc8073ae0 0x7f4fc81f4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.328+0000 7f4fc759e640 1 -- 172.21.15.98:0/2556607235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc8154bf0 msgr2=0x7f4fc81598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:13.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.328+0000 7f4fc759e640 1 --2- 172.21.15.98:0/2556607235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc8154bf0 0x7f4fc81598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.328+0000 7f4fc759e640 1 -- 172.21.15.98:0/2556607235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc815a330 con 0x7f4fc815b930 2024-09-10T20:59:13.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.328+0000 7f4fc759e640 1 --2- 172.21.15.98:0/2556607235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc815b930 0x7f4fc8159df0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009d30 tx=0x7f4fc0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:13.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.329+0000 7f4fbf7fe640 1 -- 172.21.15.98:0/2556607235 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0004550 con 0x7f4fc815b930 2024-09-10T20:59:13.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.329+0000 7f4fbf7fe640 1 -- 172.21.15.98:0/2556607235 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fc00047e0 con 0x7f4fc815b930 2024-09-10T20:59:13.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.329+0000 7f4fcc802640 1 -- 172.21.15.98:0/2556607235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc814d2b0 con 0x7f4fc815b930 2024-09-10T20:59:13.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.329+0000 7f4fbf7fe640 1 -- 172.21.15.98:0/2556607235 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc00396d0 con 0x7f4fc815b930 2024-09-10T20:59:13.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.329+0000 7f4fcc802640 1 -- 172.21.15.98:0/2556607235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc814d870 con 0x7f4fc815b930 2024-09-10T20:59:13.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.330+0000 7f4fcc802640 1 -- 172.21.15.98:0/2556607235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc80b3920 con 0x7f4fc815b930 2024-09-10T20:59:13.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.331+0000 7f4fbf7fe640 1 -- 172.21.15.98:0/2556607235 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4fc004c030 con 0x7f4fc815b930 2024-09-10T20:59:13.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.331+0000 7f4fbf7fe640 1 --2- 172.21.15.98:0/2556607235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4fa8077fa0 0x7f4fa807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:13.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.331+0000 7f4fbf7fe640 1 -- 172.21.15.98:0/2556607235 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4fc00bdb20 con 0x7f4fc815b930 2024-09-10T20:59:13.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.332+0000 7f4fc6d9d640 1 --2- 172.21.15.98:0/2556607235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4fa8077fa0 0x7f4fa807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:13.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.332+0000 7f4fc6d9d640 1 --2- 172.21.15.98:0/2556607235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4fa8077fa0 0x7f4fa807a460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8000f60 tx=0x7f4fb8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:13.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.336+0000 7f4fbf7fe640 1 -- 172.21.15.98:0/2556607235 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4fc0085c70 con 0x7f4fc815b930 2024-09-10T20:59:13.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.510+0000 7f4fcc802640 1 -- 172.21.15.98:0/2556607235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4fc806fa10 con 0x7f4fc815b930 2024-09-10T20:59:13.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.511+0000 7f4fbf7fe640 1 -- 172.21.15.98:0/2556607235 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18610 (secure 0 0 0) 0x7f4fc008a6a0 con 0x7f4fc815b930 2024-09-10T20:59:13.515 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:13.515 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":25,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:58:51.602367+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:58:44.390290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T20:58:45.564099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T20:58:49.250508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:59:13.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.513+0000 7f4fcc802640 1 -- 172.21.15.98:0/2556607235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4fa8077fa0 msgr2=0x7f4fa807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.513+0000 7f4fcc802640 1 --2- 172.21.15.98:0/2556607235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4fa8077fa0 0x7f4fa807a460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8000f60 tx=0x7f4fb8009040 comp rx=0 tx=0).stop 2024-09-10T20:59:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.513+0000 7f4fcc802640 1 -- 172.21.15.98:0/2556607235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc815b930 msgr2=0x7f4fc8159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.513+0000 7f4fcc802640 1 --2- 172.21.15.98:0/2556607235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc815b930 0x7f4fc8159df0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009d30 tx=0x7f4fc0003f90 comp rx=0 tx=0).stop 2024-09-10T20:59:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.513+0000 7f4fcc802640 1 -- 172.21.15.98:0/2556607235 shutdown_connections 2024-09-10T20:59:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.513+0000 7f4fcc802640 1 --2- 172.21.15.98:0/2556607235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4fa8077fa0 0x7f4fa807a460 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.513+0000 7f4fcc802640 1 --2- 172.21.15.98:0/2556607235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fc815b930 0x7f4fc8159df0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.513+0000 7f4fcc802640 1 --2- 172.21.15.98:0/2556607235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc8154bf0 0x7f4fc81598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.513+0000 7f4fcc802640 1 --2- 172.21.15.98:0/2556607235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc8073ae0 0x7f4fc81f4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.513+0000 7f4fcc802640 1 -- 172.21.15.98:0/2556607235 >> 172.21.15.98:0/2556607235 conn(0x7f4fc80d0970 msgr2=0x7f4fc80733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.513+0000 7f4fcc802640 1 -- 172.21.15.98:0/2556607235 shutdown_connections 2024-09-10T20:59:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.513+0000 7f4fcc802640 1 -- 172.21.15.98:0/2556607235 wait complete. 2024-09-10T20:59:13.529 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:59:13.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.625+0000 7fd8a5040640 1 -- 172.21.15.98:0/2627329327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8a00694d0 msgr2=0x7fd8a00d02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:13.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.625+0000 7fd8a5040640 1 --2- 172.21.15.98:0/2627329327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8a00694d0 0x7fd8a00d02b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd888009830 tx=0x7fd88802a5c0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.625+0000 7fd8a5040640 1 -- 172.21.15.98:0/2627329327 shutdown_connections 2024-09-10T20:59:13.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.625+0000 7fd8a5040640 1 --2- 172.21.15.98:0/2627329327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8a00694d0 0x7fd8a00d02b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.625+0000 7fd8a5040640 1 -- 172.21.15.98:0/2627329327 >> 172.21.15.98:0/2627329327 conn(0x7fd8a0074150 msgr2=0x7fd8a0074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:13.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.625+0000 7fd8a5040640 1 -- 172.21.15.98:0/2627329327 shutdown_connections 2024-09-10T20:59:13.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.625+0000 7fd8a5040640 1 -- 172.21.15.98:0/2627329327 wait complete. 2024-09-10T20:59:13.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.627+0000 7fd8a5040640 1 Processor -- start 2024-09-10T20:59:13.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.627+0000 7fd8a5040640 1 -- start start 2024-09-10T20:59:13.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd8a5040640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8a00694d0 0x7fd8a009bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:13.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd8a5040640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8a00d07f0 0x7fd8a00c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:13.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd8a5040640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a015f230 0x7fd8a00c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:13.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd8a5040640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a00b11d0 con 0x7fd8a00d07f0 2024-09-10T20:59:13.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd8a5040640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a00b1050 con 0x7fd8a00694d0 2024-09-10T20:59:13.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd8a5040640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a00b1350 con 0x7fd8a015f230 2024-09-10T20:59:13.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd89659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8a00d07f0 0x7fd8a00c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:13.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd89659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8a00d07f0 0x7fd8a00c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55512/0 (socket says 172.21.15.98:55512) 2024-09-10T20:59:13.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd89659c640 1 -- 172.21.15.98:0/2266939327 learned_addr learned my addr 172.21.15.98:0/2266939327 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:13.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd89ed9d640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8a00694d0 0x7fd8a009bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:13.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd89f59e640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a015f230 0x7fd8a00c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:13.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd89659c640 1 -- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a015f230 msgr2=0x7fd8a00c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:13.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd89659c640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a015f230 0x7fd8a00c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd89659c640 1 -- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8a00694d0 msgr2=0x7fd8a009bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:13.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd89659c640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8a00694d0 0x7fd8a009bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd89659c640 1 -- 172.21.15.98:0/2266939327 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a009c560 con 0x7fd8a00d07f0 2024-09-10T20:59:13.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd89f59e640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a015f230 0x7fd8a00c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:13.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd89ed9d640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8a00694d0 0x7fd8a009bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:13.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd89659c640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8a00d07f0 0x7fd8a00c9e70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd888009950 tx=0x7fd888032c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:13.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd897fff640 1 -- 172.21.15.98:0/2266939327 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888040070 con 0x7fd8a00d07f0 2024-09-10T20:59:13.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd897fff640 1 -- 172.21.15.98:0/2266939327 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd888005310 con 0x7fd8a00d07f0 2024-09-10T20:59:13.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd8a5040640 1 -- 172.21.15.98:0/2266939327 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a009c840 con 0x7fd8a00d07f0 2024-09-10T20:59:13.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.628+0000 7fd897fff640 1 -- 172.21.15.98:0/2266939327 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8880047a0 con 0x7fd8a00d07f0 2024-09-10T20:59:13.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.629+0000 7fd8a5040640 1 -- 172.21.15.98:0/2266939327 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a009ce00 con 0x7fd8a00d07f0 2024-09-10T20:59:13.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.629+0000 7fd8a5040640 1 -- 172.21.15.98:0/2266939327 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a00b1800 con 0x7fd8a00d07f0 2024-09-10T20:59:13.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.630+0000 7fd897fff640 1 -- 172.21.15.98:0/2266939327 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd888004940 con 0x7fd8a00d07f0 2024-09-10T20:59:13.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.631+0000 7fd897fff640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd880077fa0 0x7fd88007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:13.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.631+0000 7fd897fff640 1 -- 172.21.15.98:0/2266939327 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd8880bb680 con 0x7fd8a00d07f0 2024-09-10T20:59:13.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.631+0000 7fd89ed9d640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd880077fa0 0x7fd88007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:13.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.631+0000 7fd89ed9d640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd880077fa0 0x7fd88007a460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd890000f60 tx=0x7fd890009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:13.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.635+0000 7fd897fff640 1 -- 172.21.15.98:0/2266939327 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd888083dc0 con 0x7fd8a00d07f0 2024-09-10T20:59:13.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.843+0000 7fd8a5040640 1 -- 172.21.15.98:0/2266939327 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8a009e500 con 0x7fd8a00d07f0 2024-09-10T20:59:13.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.843+0000 7fd897fff640 1 -- 172.21.15.98:0/2266939327 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7fd8880836c0 con 0x7fd8a00d07f0 2024-09-10T20:59:13.848 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:13.848 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"btime":"2024-09-10T20:59:06:294252+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4277,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/1954552281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1954552281},{"type":"v1","addr":"172.21.15.190:6836","nonce":1954552281}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/8997965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":8997965},{"type":"v1","addr":"172.21.15.190:6837","nonce":8997965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1984550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1984550306},{"type":"v1","addr":"172.21.15.98:6833","nonce":1984550306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:58:46.560479+0000","modified":"2024-09-10T20:59:05.635381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3141703317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3141703317},{"type":"v1","addr":"172.21.15.98:6835","nonce":3141703317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T20:59:13.848 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2024-09-10T20:59:13.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.846+0000 7fd8a5040640 1 -- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd880077fa0 msgr2=0x7fd88007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:13.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.846+0000 7fd8a5040640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd880077fa0 0x7fd88007a460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd890000f60 tx=0x7fd890009040 comp rx=0 tx=0).stop 2024-09-10T20:59:13.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.846+0000 7fd8a5040640 1 -- 172.21.15.98:0/2266939327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8a00d07f0 msgr2=0x7fd8a00c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:13.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.846+0000 7fd8a5040640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8a00d07f0 0x7fd8a00c9e70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd888009950 tx=0x7fd888032c00 comp rx=0 tx=0).stop 2024-09-10T20:59:13.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.846+0000 7fd8a5040640 1 -- 172.21.15.98:0/2266939327 shutdown_connections 2024-09-10T20:59:13.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.846+0000 7fd8a5040640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd880077fa0 0x7fd88007a460 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.846+0000 7fd8a5040640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a015f230 0x7fd8a00c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.846+0000 7fd8a5040640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8a00d07f0 0x7fd8a00c9e70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.846+0000 7fd8a5040640 1 --2- 172.21.15.98:0/2266939327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8a00694d0 0x7fd8a009bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:13.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.846+0000 7fd8a5040640 1 -- 172.21.15.98:0/2266939327 >> 172.21.15.98:0/2266939327 conn(0x7fd8a0074150 msgr2=0x7fd8a00b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:13.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.846+0000 7fd8a5040640 1 -- 172.21.15.98:0/2266939327 shutdown_connections 2024-09-10T20:59:13.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:13.846+0000 7fd8a5040640 1 -- 172.21.15.98:0/2266939327 wait complete. 2024-09-10T20:59:14.205 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T20:59:14.207 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T20:59:14.264 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T20:58:46.560479+0000', 'modified': '2024-09-10T20:59:05.635381+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4380}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4380': {'gid': 4380, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6835/3141703317', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 3141703317}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 3141703317}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4380, 'qdb_cluster': [4380]} 2024-09-10T20:59:14.264 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T20:59:14.264 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T20:59:14.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.364+0000 7f925ea79640 1 -- 172.21.15.98:0/2701277528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9244003fc0 msgr2=0x7f9244004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:14.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.364+0000 7f925ea79640 1 --2- 172.21.15.98:0/2701277528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9244003fc0 0x7f9244004800 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f924800c610 tx=0x7f9244005830 comp rx=0 tx=0).stop 2024-09-10T20:59:14.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.364+0000 7f925d016640 1 -- 172.21.15.98:0/2701277528 reap_dead start 2024-09-10T20:59:14.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.364+0000 7f925ea79640 1 -- 172.21.15.98:0/2701277528 shutdown_connections 2024-09-10T20:59:14.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.364+0000 7f925ea79640 1 -- 172.21.15.98:0/2701277528 >> 172.21.15.98:0/2701277528 conn(0x7f92580b1760 msgr2=0x7f92580b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:14.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.364+0000 7f925ea79640 1 -- 172.21.15.98:0/2701277528 shutdown_connections 2024-09-10T20:59:14.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.364+0000 7f925ea79640 1 -- 172.21.15.98:0/2701277528 wait complete. 2024-09-10T20:59:14.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.365+0000 7f925ea79640 1 Processor -- start 2024-09-10T20:59:14.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.365+0000 7f925ea79640 1 -- start start 2024-09-10T20:59:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925ea79640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9244003fc0 0x7f925813aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925ea79640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9258138da0 0x7f9258135890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925ea79640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9258135dd0 0x7f92580b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925ea79640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f92580702d0 con 0x7f9258135dd0 2024-09-10T20:59:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925ea79640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9258070150 con 0x7f9244003fc0 2024-09-10T20:59:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925ea79640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9258164f20 con 0x7f9258138da0 2024-09-10T20:59:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925d016640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9258135dd0 0x7f92580b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925d016640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9258135dd0 0x7f92580b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55530/0 (socket says 172.21.15.98:55530) 2024-09-10T20:59:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925d016640 1 -- 172.21.15.98:0/1178030196 learned_addr learned my addr 172.21.15.98:0/1178030196 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925c815640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9244003fc0 0x7f925813aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f9257fff640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9258138da0 0x7f9258135890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925d016640 1 -- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9258138da0 msgr2=0x7f9258135890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925d016640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9258138da0 0x7f9258135890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925d016640 1 -- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9244003fc0 msgr2=0x7f925813aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925d016640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9244003fc0 0x7f925813aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925d016640 1 -- 172.21.15.98:0/1178030196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9258136870 con 0x7f9258135dd0 2024-09-10T20:59:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925c815640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9244003fc0 0x7f925813aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:14.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f925d016640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9258135dd0 0x7f92580b3630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9258072250 tx=0x7f9250011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:14.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.366+0000 7f92557fa640 1 -- 172.21.15.98:0/1178030196 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925000f8c0 con 0x7f9258135dd0 2024-09-10T20:59:14.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.367+0000 7f92557fa640 1 -- 172.21.15.98:0/1178030196 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f925000fa80 con 0x7f9258135dd0 2024-09-10T20:59:14.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.367+0000 7f925ea79640 1 -- 172.21.15.98:0/1178030196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9258136b50 con 0x7f9258135dd0 2024-09-10T20:59:14.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.367+0000 7f92557fa640 1 -- 172.21.15.98:0/1178030196 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250005020 con 0x7f9258135dd0 2024-09-10T20:59:14.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.367+0000 7f925ea79640 1 -- 172.21.15.98:0/1178030196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9258136520 con 0x7f9258135dd0 2024-09-10T20:59:14.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.368+0000 7f92557fa640 1 -- 172.21.15.98:0/1178030196 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9250018030 con 0x7f9258135dd0 2024-09-10T20:59:14.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.369+0000 7f92557fa640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9238077fa0 0x7f923807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.369+0000 7f92557fa640 1 -- 172.21.15.98:0/1178030196 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f925009a5d0 con 0x7f9258135dd0 2024-09-10T20:59:14.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.369+0000 7f925c815640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9238077fa0 0x7f923807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.369+0000 7f925c815640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9238077fa0 0x7f923807a460 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9240000c00 tx=0x7f9240006b10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.369+0000 7f9261848740 1 Processor -- start 2024-09-10T20:59:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.369+0000 7f9261848740 1 -- start start 2024-09-10T20:59:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f9261848740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e641b0 0x5583e5e3bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f9261848740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e83df0 0x5583e5e841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f9261848740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e395f0 0x5583e5e39a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f9261848740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5583e5db0820 con 0x5583e5e395f0 2024-09-10T20:59:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f9261848740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5583e5d9fc50 con 0x5583e5e83df0 2024-09-10T20:59:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f9261848740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x5583e5da3a60 con 0x5583e5e641b0 2024-09-10T20:59:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f925d016640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e395f0 0x5583e5e39a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f925d016640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e395f0 0x5583e5e39a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55534/0 (socket says 172.21.15.98:55534) 2024-09-10T20:59:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f925d016640 1 -- 172.21.15.98:0/2353103151 learned_addr learned my addr 172.21.15.98:0/2353103151 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f9257fff640 1 --2- 172.21.15.98:0/2353103151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e83df0 0x5583e5e841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f925c815640 1 --2- 172.21.15.98:0/2353103151 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e641b0 0x5583e5e3bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f925d016640 1 -- 172.21.15.98:0/2353103151 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e641b0 msgr2=0x5583e5e3bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f925d016640 1 --2- 172.21.15.98:0/2353103151 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e641b0 0x5583e5e3bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f925d016640 1 -- 172.21.15.98:0/2353103151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e83df0 msgr2=0x5583e5e841f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f925d016640 1 --2- 172.21.15.98:0/2353103151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e83df0 0x5583e5e841f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.370+0000 7f925d016640 1 -- 172.21.15.98:0/2353103151 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e5b01850 con 0x5583e5e395f0 2024-09-10T20:59:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.371+0000 7f925d016640 1 --2- 172.21.15.98:0/2353103151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e395f0 0x5583e5e39a40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f925005f990 tx=0x7f9250061f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb6a683e2047fff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.371+0000 7f924e7fc640 1 -- 172.21.15.98:0/2353103151 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250067200 con 0x5583e5e395f0 2024-09-10T20:59:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.371+0000 7f924e7fc640 1 -- 172.21.15.98:0/2353103151 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f925006bc20 con 0x5583e5e395f0 2024-09-10T20:59:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.371+0000 7f924e7fc640 1 -- 172.21.15.98:0/2353103151 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250098de0 con 0x5583e5e395f0 2024-09-10T20:59:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.371+0000 7f9261848740 1 -- 172.21.15.98:0/2353103151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e395f0 msgr2=0x5583e5e39a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.371+0000 7f9261848740 1 --2- 172.21.15.98:0/2353103151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e395f0 0x5583e5e39a40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f925005f990 tx=0x7f9250061f50 comp rx=0 tx=0).stop 2024-09-10T20:59:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.371+0000 7f9261848740 1 -- 172.21.15.98:0/2353103151 shutdown_connections 2024-09-10T20:59:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.371+0000 7f9261848740 1 --2- 172.21.15.98:0/2353103151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e395f0 0x5583e5e39a40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.371+0000 7f9261848740 1 --2- 172.21.15.98:0/2353103151 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e83df0 0x5583e5e841f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.371+0000 7f9261848740 1 --2- 172.21.15.98:0/2353103151 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e641b0 0x5583e5e3bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.371+0000 7f9261848740 1 -- 172.21.15.98:0/2353103151 >> 172.21.15.98:0/2353103151 conn(0x5583e5e3a6a0 msgr2=0x5583e5e3e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.371+0000 7f9261848740 1 -- 172.21.15.98:0/2353103151 shutdown_connections 2024-09-10T20:59:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.371+0000 7f9261848740 1 -- 172.21.15.98:0/2353103151 wait complete. 2024-09-10T20:59:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.372+0000 7f9261848740 1 -- start start 2024-09-10T20:59:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.372+0000 7f9261848740 1 Processor -- start 2024-09-10T20:59:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9261848740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e641b0 0x5583e5e96d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9261848740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e83df0 0x5583e5e58c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9261848740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e5edf0 0x5583e5e680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9261848740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5583e5b10620 con 0x5583e5e83df0 2024-09-10T20:59:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9261848740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5583e5d455f0 con 0x5583e5e641b0 2024-09-10T20:59:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9261848740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x5583e5e38b30 con 0x5583e5e5edf0 2024-09-10T20:59:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9261848740 10 client.0 _ll_register_callbacks cb 0x5583e5d490a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T20:59:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9257fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e83df0 0x5583e5e58c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9257fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e83df0 0x5583e5e58c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55536/0 (socket says 172.21.15.98:55536) 2024-09-10T20:59:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9257fff640 1 -- 172.21.15.98:0/3322332519 learned_addr learned my addr 172.21.15.98:0/3322332519 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f925c815640 1 --2- 172.21.15.98:0/3322332519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e641b0 0x5583e5e96d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f925d016640 1 --2- 172.21.15.98:0/3322332519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e5edf0 0x5583e5e680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9257fff640 1 -- 172.21.15.98:0/3322332519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e5edf0 msgr2=0x5583e5e680e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9257fff640 1 --2- 172.21.15.98:0/3322332519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e5edf0 0x5583e5e680e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9257fff640 1 -- 172.21.15.98:0/3322332519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e641b0 msgr2=0x5583e5e96d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9257fff640 1 --2- 172.21.15.98:0/3322332519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e641b0 0x5583e5e96d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9257fff640 1 -- 172.21.15.98:0/3322332519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e5b786b0 con 0x5583e5e83df0 2024-09-10T20:59:14.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f9257fff640 1 --2- 172.21.15.98:0/3322332519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e83df0 0x5583e5e58c30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f924800c610 tx=0x7f9248001560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a869425e8e892f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:14.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.373+0000 7f92377fe640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-10T20:59:14.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f92377fe640 1 -- 172.21.15.98:0/3322332519 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248044060 con 0x5583e5e83df0 2024-09-10T20:59:14.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f92377fe640 1 -- 172.21.15.98:0/3322332519 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f924803fdf0 con 0x5583e5e83df0 2024-09-10T20:59:14.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f9261848740 1 -- 172.21.15.98:0/3322332519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f925006bc20 con 0x5583e5e83df0 2024-09-10T20:59:14.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f92377fe640 1 -- 172.21.15.98:0/3322332519 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f9248003b70 con 0x5583e5e83df0 2024-09-10T20:59:14.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f9261848740 10 client.4803 fetch_fsmap learned FSMap version 8 2024-09-10T20:59:14.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f9261848740 1 -- 172.21.15.98:0/3322332519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5583e5e659b0 con 0x5583e5e83df0 2024-09-10T20:59:14.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f92377fe640 1 -- 172.21.15.98:0/3322332519 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f925006bc20 con 0x5583e5e83df0 2024-09-10T20:59:14.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f92377fe640 1 -- 172.21.15.98:0/3322332519 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 8) ==== 3510+0+0 (secure 0 0 0) 0x7f924802b6e0 con 0x5583e5e83df0 2024-09-10T20:59:14.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f9261848740 10 client.4803 fetch_fsmap finished waiting for FSMap version 8 2024-09-10T20:59:14.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f9261848740 10 client.4803 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T20:59:14.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f9261848740 20 client.4803 populate_metadata read hostname 'smithi098' 2024-09-10T20:59:14.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f9261848740 1 --2- 172.21.15.98:0/3322332519 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x5583e5e5d570 0x5583e5e66090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f9261848740 4 client.4803 mds_command: new command op to 4380 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T20:59:14.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f9261848740 1 -- 172.21.15.98:0/3322332519 --> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583e5db2770 con 0x5583e5e5d570 2024-09-10T20:59:14.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.374+0000 7f925c815640 1 --2- 172.21.15.98:0/3322332519 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x5583e5e5d570 0x5583e5e66090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f92377fe640 10 client.4803 ms_handle_connect on v2:172.21.15.98:6834/3141703317 2024-09-10T20:59:14.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f925c815640 1 --2- 172.21.15.98:0/3322332519 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x5583e5e5d570 0x5583e5e66090 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac6ad0e8ffca5c66 server_cookie=a31de058707295f1 in_seq=0 out_seq=0 2024-09-10T20:59:14.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f92377fe640 1 -- 172.21.15.98:0/3322332519 <== mds.0 v2:172.21.15.98:6834/3141703317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5583e5db2770 con 0x5583e5e5d570 2024-09-10T20:59:14.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f92377fe640 10 client.4803 handle_command_reply: tid=0 2024-09-10T20:59:14.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f9261848740 1 client.4803 shutdown 2024-09-10T20:59:14.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f9261848740 1 -- 172.21.15.98:0/3322332519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e83df0 msgr2=0x5583e5e58c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:14.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f9261848740 1 --2- 172.21.15.98:0/3322332519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e83df0 0x5583e5e58c30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f924800c610 tx=0x7f9248001560 comp rx=0 tx=0).stop 2024-09-10T20:59:14.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f9261848740 1 -- 172.21.15.98:0/3322332519 shutdown_connections 2024-09-10T20:59:14.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f9261848740 1 --2- 172.21.15.98:0/3322332519 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x5583e5e5d570 0x5583e5e66090 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f9261848740 1 --2- 172.21.15.98:0/3322332519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e5edf0 0x5583e5e680e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f9261848740 1 --2- 172.21.15.98:0/3322332519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e83df0 0x5583e5e58c30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f9261848740 1 --2- 172.21.15.98:0/3322332519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e641b0 0x5583e5e96d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f92377fe640 0 client.4803 ms_handle_reset on v2:172.21.15.98:6834/3141703317 2024-09-10T20:59:14.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f9261848740 1 -- 172.21.15.98:0/3322332519 >> 172.21.15.98:0/3322332519 conn(0x5583e5e395f0 msgr2=0x5583e5e3e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:14.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f9261848740 1 -- 172.21.15.98:0/3322332519 shutdown_connections 2024-09-10T20:59:14.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f9261848740 1 -- 172.21.15.98:0/3322332519 wait complete. 2024-09-10T20:59:14.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.375+0000 7f9261848740 20 client.4803 trim_cache size 0 max 16384 2024-09-10T20:59:14.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.393+0000 7f9261848740 1 Processor -- start 2024-09-10T20:59:14.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.393+0000 7f9261848740 1 -- start start 2024-09-10T20:59:14.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f9261848740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e5edf0 0x5583e5e925d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f9261848740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e641b0 0x5583e5e844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f9261848740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e83df0 0x5583e5e83730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f9261848740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9258202bc0 con 0x5583e5e641b0 2024-09-10T20:59:14.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f9261848740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9258201c00 con 0x5583e5e5edf0 2024-09-10T20:59:14.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f9261848740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f92582015b0 con 0x5583e5e83df0 2024-09-10T20:59:14.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f9257fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e641b0 0x5583e5e844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f9257fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e641b0 0x5583e5e844b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55552/0 (socket says 172.21.15.98:55552) 2024-09-10T20:59:14.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f9257fff640 1 -- 172.21.15.98:0/1969038508 learned_addr learned my addr 172.21.15.98:0/1969038508 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:14.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f925c815640 1 --2- 172.21.15.98:0/1969038508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e5edf0 0x5583e5e925d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f925d016640 1 --2- 172.21.15.98:0/1969038508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e83df0 0x5583e5e83730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f9257fff640 1 -- 172.21.15.98:0/1969038508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e83df0 msgr2=0x5583e5e83730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:14.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f9257fff640 1 --2- 172.21.15.98:0/1969038508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e83df0 0x5583e5e83730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f9257fff640 1 -- 172.21.15.98:0/1969038508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e5edf0 msgr2=0x5583e5e925d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:14.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f9257fff640 1 --2- 172.21.15.98:0/1969038508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e5edf0 0x5583e5e925d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.394+0000 7f9257fff640 1 -- 172.21.15.98:0/1969038508 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e5e5fb90 con 0x5583e5e641b0 2024-09-10T20:59:14.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.395+0000 7f9257fff640 1 --2- 172.21.15.98:0/1969038508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e641b0 0x5583e5e844b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9248007010 tx=0x7f92480014d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9311ddac279b7a1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:14.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.395+0000 7f924dffb640 1 -- 172.21.15.98:0/1969038508 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9248003970 con 0x5583e5e641b0 2024-09-10T20:59:14.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.395+0000 7f924dffb640 1 -- 172.21.15.98:0/1969038508 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f924803a040 con 0x5583e5e641b0 2024-09-10T20:59:14.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.395+0000 7f924dffb640 1 -- 172.21.15.98:0/1969038508 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924802bbe0 con 0x5583e5e641b0 2024-09-10T20:59:14.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.395+0000 7f9261848740 1 -- 172.21.15.98:0/1969038508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e641b0 msgr2=0x5583e5e844b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:14.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.395+0000 7f9261848740 1 --2- 172.21.15.98:0/1969038508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e641b0 0x5583e5e844b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9248007010 tx=0x7f92480014d0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.395+0000 7f9261848740 1 -- 172.21.15.98:0/1969038508 shutdown_connections 2024-09-10T20:59:14.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.395+0000 7f9261848740 1 --2- 172.21.15.98:0/1969038508 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e83df0 0x5583e5e83730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.395+0000 7f9261848740 1 --2- 172.21.15.98:0/1969038508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e641b0 0x5583e5e844b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.395+0000 7f9261848740 1 --2- 172.21.15.98:0/1969038508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e5edf0 0x5583e5e925d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.395+0000 7f9261848740 1 -- 172.21.15.98:0/1969038508 >> 172.21.15.98:0/1969038508 conn(0x5583e5e395f0 msgr2=0x5583e5e39ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:14.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.395+0000 7f9261848740 1 -- 172.21.15.98:0/1969038508 shutdown_connections 2024-09-10T20:59:14.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.395+0000 7f9261848740 1 -- 172.21.15.98:0/1969038508 wait complete. 2024-09-10T20:59:14.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.396+0000 7f9261848740 1 -- start start 2024-09-10T20:59:14.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.396+0000 7f9261848740 1 Processor -- start 2024-09-10T20:59:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f9261848740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e5edf0 0x5583e5e33be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f9261848740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e641b0 0x5583e5f03790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f9261848740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e83df0 0x5583e5e3def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f9261848740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5583e5e5d3f0 con 0x5583e5e5edf0 2024-09-10T20:59:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f9261848740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5583e5e5fe60 con 0x5583e5e83df0 2024-09-10T20:59:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f9261848740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x5583e5e5d9e0 con 0x5583e5e641b0 2024-09-10T20:59:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f9261848740 10 client.0 _ll_register_callbacks cb 0x7f9250005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T20:59:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f925c815640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e5edf0 0x5583e5e33be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f925c815640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e5edf0 0x5583e5e33be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55564/0 (socket says 172.21.15.98:55564) 2024-09-10T20:59:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f925c815640 1 -- 172.21.15.98:0/2115167345 learned_addr learned my addr 172.21.15.98:0/2115167345 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f925d016640 1 --2- 172.21.15.98:0/2115167345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e83df0 0x5583e5e3def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f925c815640 1 -- 172.21.15.98:0/2115167345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e641b0 msgr2=0x5583e5f03790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f925c815640 1 --2- 172.21.15.98:0/2115167345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e641b0 0x5583e5f03790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f925c815640 1 -- 172.21.15.98:0/2115167345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e83df0 msgr2=0x5583e5e3def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f925c815640 1 --2- 172.21.15.98:0/2115167345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e83df0 0x5583e5e3def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f925c815640 1 -- 172.21.15.98:0/2115167345 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e5f078d0 con 0x5583e5e5edf0 2024-09-10T20:59:14.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f925d016640 1 --2- 172.21.15.98:0/2115167345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e83df0 0x5583e5e3def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:14.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f925c815640 1 --2- 172.21.15.98:0/2115167345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e5edf0 0x5583e5e33be0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9240006170 tx=0x7f9240014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a60e91117496b66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:14.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f92377fe640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-10T20:59:14.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f92377fe640 1 -- 172.21.15.98:0/2115167345 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9240020e60 con 0x5583e5e5edf0 2024-09-10T20:59:14.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f92377fe640 1 -- 172.21.15.98:0/2115167345 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9240032650 con 0x5583e5e5edf0 2024-09-10T20:59:14.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.397+0000 7f9261848740 1 -- 172.21.15.98:0/2115167345 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f924803a040 con 0x5583e5e5edf0 2024-09-10T20:59:14.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.398+0000 7f92377fe640 1 -- 172.21.15.98:0/2115167345 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f924003a050 con 0x5583e5e5edf0 2024-09-10T20:59:14.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.398+0000 7f9261848740 10 client.4809 fetch_fsmap learned FSMap version 8 2024-09-10T20:59:14.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.398+0000 7f9261848740 1 -- 172.21.15.98:0/2115167345 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5583e5f03cd0 con 0x5583e5e5edf0 2024-09-10T20:59:14.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.398+0000 7f92377fe640 1 -- 172.21.15.98:0/2115167345 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f924803a040 con 0x5583e5e5edf0 2024-09-10T20:59:14.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.398+0000 7f92377fe640 1 -- 172.21.15.98:0/2115167345 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 8) ==== 3510+0+0 (secure 0 0 0) 0x7f924003d020 con 0x5583e5e5edf0 2024-09-10T20:59:14.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.398+0000 7f9261848740 10 client.4809 fetch_fsmap finished waiting for FSMap version 8 2024-09-10T20:59:14.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.398+0000 7f9261848740 10 client.4809 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T20:59:14.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.398+0000 7f9261848740 20 client.4809 populate_metadata read hostname 'smithi098' 2024-09-10T20:59:14.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.398+0000 7f9261848740 1 --2- 172.21.15.98:0/2115167345 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x5583e5e7b880 0x5583e5e65e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:14.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.398+0000 7f9261848740 4 client.4809 mds_command: new command op to 4380 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T20:59:14.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.398+0000 7f9261848740 1 -- 172.21.15.98:0/2115167345 --> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] -- command(tid 0: {"prefix": "status"}) -- 0x5583e5dae220 con 0x5583e5e7b880 2024-09-10T20:59:14.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.398+0000 7f9257fff640 1 --2- 172.21.15.98:0/2115167345 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x5583e5e7b880 0x5583e5e65e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:14.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.398+0000 7f92377fe640 10 client.4809 ms_handle_connect on v2:172.21.15.98:6834/3141703317 2024-09-10T20:59:14.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.398+0000 7f9257fff640 1 --2- 172.21.15.98:0/2115167345 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x5583e5e7b880 0x5583e5e65e20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2abc4276b095c52 server_cookie=e910ed2c88461591 in_seq=0 out_seq=0 2024-09-10T20:59:14.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f92377fe640 1 -- 172.21.15.98:0/2115167345 <== mds.0 v2:172.21.15.98:6834/3141703317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5583e5dae220 con 0x5583e5e7b880 2024-09-10T20:59:14.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f92377fe640 10 client.4809 handle_command_reply: tid=0 2024-09-10T20:59:14.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f9261848740 1 client.4809 shutdown 2024-09-10T20:59:14.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f9261848740 1 -- 172.21.15.98:0/2115167345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e5edf0 msgr2=0x5583e5e33be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:14.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f9261848740 1 --2- 172.21.15.98:0/2115167345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e5edf0 0x5583e5e33be0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9240006170 tx=0x7f9240014070 comp rx=0 tx=0).stop 2024-09-10T20:59:14.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f9261848740 1 -- 172.21.15.98:0/2115167345 shutdown_connections 2024-09-10T20:59:14.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f9261848740 1 --2- 172.21.15.98:0/2115167345 >> [v2:172.21.15.98:6834/3141703317,v1:172.21.15.98:6835/3141703317] conn(0x5583e5e7b880 0x5583e5e65e20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f9261848740 1 --2- 172.21.15.98:0/2115167345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5583e5e83df0 0x5583e5e3def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f9261848740 1 --2- 172.21.15.98:0/2115167345 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5583e5e641b0 0x5583e5f03790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f9261848740 1 --2- 172.21.15.98:0/2115167345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583e5e5edf0 0x5583e5e33be0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f92377fe640 0 client.4809 ms_handle_reset on v2:172.21.15.98:6834/3141703317 2024-09-10T20:59:14.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f9261848740 1 -- 172.21.15.98:0/2115167345 >> 172.21.15.98:0/2115167345 conn(0x5583e5e395f0 msgr2=0x5583e5e83520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:14.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f9261848740 1 -- 172.21.15.98:0/2115167345 shutdown_connections 2024-09-10T20:59:14.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f9261848740 1 -- 172.21.15.98:0/2115167345 wait complete. 2024-09-10T20:59:14.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f9261848740 20 client.4809 trim_cache size 0 max 16384 2024-09-10T20:59:14.411 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-10T20:59:14.411 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "41041377-a016-49fd-be1f-323d0b834557", 2024-09-10T20:59:14.411 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-10T20:59:14.411 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 4380, 2024-09-10T20:59:14.411 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-10T20:59:14.411 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-10T20:59:14.411 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-10T20:59:14.411 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 27.181886414000001, 2024-09-10T20:59:14.411 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 8, 2024-09-10T20:59:14.411 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 25, 2024-09-10T20:59:14.411 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 25, 2024-09-10T20:59:14.411 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 33.211098206000003 2024-09-10T20:59:14.412 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-10T20:59:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f925ea79640 1 -- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9238077fa0 msgr2=0x7f923807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f925ea79640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9238077fa0 0x7f923807a460 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9240000c00 tx=0x7f9240006b10 comp rx=0 tx=0).stop 2024-09-10T20:59:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f925ea79640 1 -- 172.21.15.98:0/1178030196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9258135dd0 msgr2=0x7f92580b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f925ea79640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9258135dd0 0x7f92580b3630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9258072250 tx=0x7f9250011bd0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f925ea79640 1 -- 172.21.15.98:0/1178030196 shutdown_connections 2024-09-10T20:59:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f925ea79640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9238077fa0 0x7f923807a460 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f925ea79640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9258135dd0 0x7f92580b3630 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f925ea79640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9258138da0 0x7f9258135890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.399+0000 7f925ea79640 1 --2- 172.21.15.98:0/1178030196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9244003fc0 0x7f925813aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.400+0000 7f925ea79640 1 -- 172.21.15.98:0/1178030196 >> 172.21.15.98:0/1178030196 conn(0x7f92580b1760 msgr2=0x7f925813d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.400+0000 7f925ea79640 1 -- 172.21.15.98:0/1178030196 shutdown_connections 2024-09-10T20:59:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:14.400+0000 7f925ea79640 1 -- 172.21.15.98:0/1178030196 wait complete. 2024-09-10T20:59:14.414 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T20:59:14.428 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T20:59:14.428 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-10T20:59:14.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:14.429 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-10T20:59:14.445 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-10T20:59:14.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:14.445 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T20:59:14.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:14.510 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link delete ceph-brx 2024-09-10T20:59:14.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:14.613 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-10T20:59:14.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:14.629 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link delete ceph-brx 2024-09-10T20:59:14.696 INFO:teuthology.orchestra.run.smithi098.stderr:Cannot find device "ceph-brx" 2024-09-10T20:59:14.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:59:14.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:14.698 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-09-10T20:59:14.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:14.718 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-09-10T20:59:14.789 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2024-09-10T20:59:14.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:59:14.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:14.791 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-09-10T20:59:14.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:14.849 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-09-10T20:59:14.919 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2024-09-10T20:59:14.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:59:14.920 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-09-10T20:59:14.921 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T20:59:14.921 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T20:59:14.921 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T20:59:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.018+0000 7fcb74df3640 1 -- 172.21.15.98:0/3257236157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb70072ad0 msgr2=0x7fcb700cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.018+0000 7fcb74df3640 1 --2- 172.21.15.98:0/3257236157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb70072ad0 0x7fcb700cf040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcb7015e6b0 tx=0x7fcb6002a4a0 comp rx=0 tx=0).stop 2024-09-10T20:59:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.018+0000 7fcb74df3640 1 -- 172.21.15.98:0/3257236157 shutdown_connections 2024-09-10T20:59:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.018+0000 7fcb74df3640 1 --2- 172.21.15.98:0/3257236157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb70072ad0 0x7fcb700cf040 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.018+0000 7fcb74df3640 1 --2- 172.21.15.98:0/3257236157 >> v2:172.21.15.190:3300/0 conn(0x7fcb70072680 0x7fcb700ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:15.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.018+0000 7fcb74df3640 1 -- 172.21.15.98:0/3257236157 >> 172.21.15.98:0/3257236157 conn(0x7fcb700b1760 msgr2=0x7fcb700b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:15.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.018+0000 7fcb74df3640 1 -- 172.21.15.98:0/3257236157 shutdown_connections 2024-09-10T20:59:15.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.018+0000 7fcb74df3640 1 -- 172.21.15.98:0/3257236157 wait complete. 2024-09-10T20:59:15.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.020+0000 7fcb74df3640 1 Processor -- start 2024-09-10T20:59:15.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.020+0000 7fcb74df3640 1 -- start start 2024-09-10T20:59:15.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb74df3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb70072680 0x7fcb7006fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:15.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb74df3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb70072ad0 0x7fcb701f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:15.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb74df3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb70144780 0x7fcb7015dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:15.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb74df3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7015c8e0 con 0x7fcb70072ad0 2024-09-10T20:59:15.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb74df3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7015c760 con 0x7fcb70072680 2024-09-10T20:59:15.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb74df3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcb7015ca60 con 0x7fcb70144780 2024-09-10T20:59:15.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb6dd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb70072ad0 0x7fcb701f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:15.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb6dd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb70072ad0 0x7fcb701f6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55584/0 (socket says 172.21.15.98:55584) 2024-09-10T20:59:15.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb6dd9b640 1 -- 172.21.15.98:0/3882721237 learned_addr learned my addr 172.21.15.98:0/3882721237 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:15.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb6e59c640 1 --2- 172.21.15.98:0/3882721237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb70072680 0x7fcb7006fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:15.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb6ed9d640 1 --2- 172.21.15.98:0/3882721237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb70144780 0x7fcb7015dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:15.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb6dd9b640 1 -- 172.21.15.98:0/3882721237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb70144780 msgr2=0x7fcb7015dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:15.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb6dd9b640 1 --2- 172.21.15.98:0/3882721237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb70144780 0x7fcb7015dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:15.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb6dd9b640 1 -- 172.21.15.98:0/3882721237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb70072680 msgr2=0x7fcb7006fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:15.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb6dd9b640 1 --2- 172.21.15.98:0/3882721237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb70072680 0x7fcb7006fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:15.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb6dd9b640 1 -- 172.21.15.98:0/3882721237 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb701f69d0 con 0x7fcb70072ad0 2024-09-10T20:59:15.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb6dd9b640 1 --2- 172.21.15.98:0/3882721237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb70072ad0 0x7fcb701f6060 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800d950 tx=0x7fcb5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:15.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb66ffd640 1 -- 172.21.15.98:0/3882721237 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb58017070 con 0x7fcb70072ad0 2024-09-10T20:59:15.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb66ffd640 1 -- 172.21.15.98:0/3882721237 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb58009d20 con 0x7fcb70072ad0 2024-09-10T20:59:15.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb66ffd640 1 -- 172.21.15.98:0/3882721237 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb580106c0 con 0x7fcb70072ad0 2024-09-10T20:59:15.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb74df3640 1 -- 172.21.15.98:0/3882721237 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb701f6cb0 con 0x7fcb70072ad0 2024-09-10T20:59:15.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.021+0000 7fcb74df3640 1 -- 172.21.15.98:0/3882721237 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb700701a0 con 0x7fcb70072ad0 2024-09-10T20:59:15.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.022+0000 7fcb74df3640 1 -- 172.21.15.98:0/3882721237 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb7015cf10 con 0x7fcb70072ad0 2024-09-10T20:59:15.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.023+0000 7fcb66ffd640 1 -- 172.21.15.98:0/3882721237 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcb58010860 con 0x7fcb70072ad0 2024-09-10T20:59:15.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.023+0000 7fcb66ffd640 1 --2- 172.21.15.98:0/3882721237 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcb50077f70 0x7fcb5007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:15.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.023+0000 7fcb66ffd640 1 -- 172.21.15.98:0/3882721237 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcb5809a460 con 0x7fcb70072ad0 2024-09-10T20:59:15.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.024+0000 7fcb6e59c640 1 --2- 172.21.15.98:0/3882721237 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcb50077f70 0x7fcb5007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:15.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.024+0000 7fcb6e59c640 1 --2- 172.21.15.98:0/3882721237 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcb50077f70 0x7fcb5007a430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcb68007fd0 tx=0x7fcb68009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:15.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.028+0000 7fcb66ffd640 1 -- 172.21.15.98:0/3882721237 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fcb58062cb0 con 0x7fcb70072ad0 2024-09-10T20:59:15.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.236+0000 7fcb74df3640 1 -- 172.21.15.98:0/3882721237 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcb701390b0 con 0x7fcb70072ad0 2024-09-10T20:59:15.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.236+0000 7fcb66ffd640 1 -- 172.21.15.98:0/3882721237 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7fcb580625b0 con 0x7fcb70072ad0 2024-09-10T20:59:15.240 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T20:59:15.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.238+0000 7fcb74df3640 1 -- 172.21.15.98:0/3882721237 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcb50077f70 msgr2=0x7fcb5007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:15.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.238+0000 7fcb74df3640 1 --2- 172.21.15.98:0/3882721237 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcb50077f70 0x7fcb5007a430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcb68007fd0 tx=0x7fcb68009040 comp rx=0 tx=0).stop 2024-09-10T20:59:15.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.238+0000 7fcb74df3640 1 -- 172.21.15.98:0/3882721237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb70072ad0 msgr2=0x7fcb701f6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:15.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.238+0000 7fcb74df3640 1 --2- 172.21.15.98:0/3882721237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb70072ad0 0x7fcb701f6060 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800d950 tx=0x7fcb5800de10 comp rx=0 tx=0).stop 2024-09-10T20:59:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.238+0000 7fcb74df3640 1 -- 172.21.15.98:0/3882721237 shutdown_connections 2024-09-10T20:59:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.238+0000 7fcb74df3640 1 --2- 172.21.15.98:0/3882721237 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcb50077f70 0x7fcb5007a430 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.238+0000 7fcb74df3640 1 --2- 172.21.15.98:0/3882721237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb70144780 0x7fcb7015dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.238+0000 7fcb74df3640 1 --2- 172.21.15.98:0/3882721237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb70072ad0 0x7fcb701f6060 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.238+0000 7fcb74df3640 1 --2- 172.21.15.98:0/3882721237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb70072680 0x7fcb7006fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.238+0000 7fcb74df3640 1 -- 172.21.15.98:0/3882721237 >> 172.21.15.98:0/3882721237 conn(0x7fcb700b1760 msgr2=0x7fcb701395d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.238+0000 7fcb74df3640 1 -- 172.21.15.98:0/3882721237 shutdown_connections 2024-09-10T20:59:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15.238+0000 7fcb74df3640 1 -- 172.21.15.98:0/3882721237 wait complete. 2024-09-10T20:59:15.253 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T20:59:15.253 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T20:59:15.253 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-10T20:59:15.253 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T20:59:15.253 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T20:59:15.253 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T20:59:15.253 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T20:59:15.253 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T20:59:15.253 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T20:59:15.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:15.254 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-10T20:59:15.269 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T20:59:15.269 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T20:59:15.269 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T20:59:15.269 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:15.269 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-10T20:59:15.269 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:15.269 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:15.269 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:15.272 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-10T20:59:15.272 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:15.272 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:15.272 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-10T20:59:15.272 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:15.272 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:15.272 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T20:59:15.272 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:15.273 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T20:59:15.273 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42769sec preferred_lft 42769sec 2024-09-10T20:59:15.273 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link noprefixroute 2024-09-10T20:59:15.273 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:15.273 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T20:59:15.273 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:15.273 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:59:15.274 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add name ceph-brx type bridge 2024-09-10T20:59:15.274 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip addr flush dev ceph-brx 2024-09-10T20:59:15.274 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set ceph-brx up 2024-09-10T20:59:15.274 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T20:59:15.274 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:59:15.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:15.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:15.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:15.407 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T20:59:15.431 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-10T20:59:15.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:15.433 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-10T20:59:15.490 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-10T20:59:15.490 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T20:59:15.490 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T20:59:15.490 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T20:59:15.490 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T20:59:15.491 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:15.491 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:59:15.491 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T20:59:15.491 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T20:59:15.491 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T20:59:15.491 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:59:15.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:15.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:15.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:15.617 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-10T20:59:15.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:15.633 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list-id 2024-09-10T20:59:15.688 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:15.689 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:59:15.689 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T20:59:15.689 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T20:59:15.689 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:59:15.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:15.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:15.805 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T20:59:15.806 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:15.806 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:59:15.806 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T20:59:15.806 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T20:59:15.806 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T20:59:15.806 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T20:59:15.806 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T20:59:15.806 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:59:15.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:15.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:15.918 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:15.918 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:59:15.918 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-10T20:59:15.918 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T20:59:15.918 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:59:15.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:15.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:15.993 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T20:59:15.993 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T20:59:16.008 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T20:59:16.009 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T20:59:16.009 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T20:59:16.064 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T20:59:16.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:16.064 DEBUG:teuthology.orchestra.run.smithi098:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:16.146 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:16.147 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T20:59:16.266 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:16.266 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T20:59:16.266 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-09-10T20:59:16.266 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: trying mount with new device syntax: 0@41041377-a016-49fd-be1f-323d0b834557.cephfs=/ 2024-09-10T20:59:16.266 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.98:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2024-09-10T20:59:16.267 INFO:teuthology.orchestra.run.smithi098.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T20:59:16.267 INFO:teuthology.orchestra.run.smithi098.stdout: You just mounted a file system that supports labels which does not 2024-09-10T20:59:16.267 INFO:teuthology.orchestra.run.smithi098.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T20:59:16.267 INFO:teuthology.orchestra.run.smithi098.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T20:59:16.267 INFO:teuthology.orchestra.run.smithi098.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T20:59:16.269 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T20:59:16.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:16.269 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T20:59:16.333 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T20:59:16.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T20:59:16.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:16.429 INFO:teuthology.orchestra.run.smithi098.stdout:client4526 2024-09-10T20:59:16.430 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:59:16.430 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:16.446 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T20:59:16.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T20:59:16.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:16.540 INFO:teuthology.orchestra.run.smithi098.stdout:client4526 2024-09-10T20:59:16.541 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T20:59:16.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T20:59:16.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:16.595 INFO:teuthology.orchestra.run.smithi098.stdout:client4526 2024-09-10T20:59:16.595 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/41041377-a016-49fd-be1f-323d0b834557.client4526/status') 2024-09-10T20:59:16.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:16.653 INFO:teuthology.orchestra.run.smithi098.stdout:instance: client.4526 (3)172.21.15.98:0/1895748602 2024-09-10T20:59:16.653 INFO:teuthology.orchestra.run.smithi098.stdout:blocklisted: false 2024-09-10T20:59:16.653 INFO:teuthology.orchestra.run.smithi098.stderr:0+1 records in 2024-09-10T20:59:16.653 INFO:teuthology.orchestra.run.smithi098.stderr:0+1 records out 2024-09-10T20:59:16.653 INFO:teuthology.orchestra.run.smithi098.stderr:70 bytes copied, 2.2888e-05 s, 3.1 MB/s 2024-09-10T20:59:16.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:16.660 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-09-10T20:59:16.661 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T20:59:16.661 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T20:59:16.661 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T20:59:16.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.758+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/1554031699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d480b0d50 msgr2=0x7f7d480af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:16.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.758+0000 7f7d4f9a3640 1 --2- 172.21.15.98:0/1554031699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d480b0d50 0x7f7d480af5b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7d48073670 tx=0x7f7d3c02a4a0 comp rx=0 tx=0).stop 2024-09-10T20:59:16.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.758+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/1554031699 shutdown_connections 2024-09-10T20:59:16.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.758+0000 7f7d4f9a3640 1 --2- 172.21.15.98:0/1554031699 >> v2:172.21.15.190:3300/0 conn(0x7f7d480cff70 0x7f7d480afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:16.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.758+0000 7f7d4f9a3640 1 --2- 172.21.15.98:0/1554031699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d480b0d50 0x7f7d480af5b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:16.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.758+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/1554031699 >> 172.21.15.98:0/1554031699 conn(0x7f7d480b37c0 msgr2=0x7f7d480b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:16.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.758+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/1554031699 shutdown_connections 2024-09-10T20:59:16.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.758+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/1554031699 wait complete. 2024-09-10T20:59:16.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.759+0000 7f7d4f9a3640 1 Processor -- start 2024-09-10T20:59:16.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.759+0000 7f7d4f9a3640 1 -- start start 2024-09-10T20:59:16.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4f9a3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d480cff70 0x7f7d481502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:16.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4f9a3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d48160530 0x7f7d481f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:16.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4f9a3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d4815c720 0x7f7d481e9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:16.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4f9a3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7d480b0030 con 0x7f7d480cff70 2024-09-10T20:59:16.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4f9a3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7d480b34f0 con 0x7f7d4815c720 2024-09-10T20:59:16.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4f9a3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7d480b01b0 con 0x7f7d48160530 2024-09-10T20:59:16.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4d73f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d480cff70 0x7f7d481502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:16.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4d73f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d480cff70 0x7f7d481502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55606/0 (socket says 172.21.15.98:55606) 2024-09-10T20:59:16.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4d73f640 1 -- 172.21.15.98:0/2245656691 learned_addr learned my addr 172.21.15.98:0/2245656691 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:16.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4d73f640 1 -- 172.21.15.98:0/2245656691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d48160530 msgr2=0x7f7d481f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:16.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4d73f640 1 --2- 172.21.15.98:0/2245656691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d48160530 0x7f7d481f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:16.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4d73f640 1 -- 172.21.15.98:0/2245656691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d4815c720 msgr2=0x7f7d481e9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:16.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4d73f640 1 --2- 172.21.15.98:0/2245656691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d4815c720 0x7f7d481e9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:16.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4d73f640 1 -- 172.21.15.98:0/2245656691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d480c6e80 con 0x7f7d480cff70 2024-09-10T20:59:16.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d4d73f640 1 --2- 172.21.15.98:0/2245656691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d480cff70 0x7f7d481502a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c7a0 tx=0x7f7d3800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:16.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d45ffb640 1 -- 172.21.15.98:0/2245656691 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38017070 con 0x7f7d480cff70 2024-09-10T20:59:16.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d45ffb640 1 -- 172.21.15.98:0/2245656691 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d38005070 con 0x7f7d480cff70 2024-09-10T20:59:16.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.760+0000 7f7d45ffb640 1 -- 172.21.15.98:0/2245656691 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d380054a0 con 0x7f7d480cff70 2024-09-10T20:59:16.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.761+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/2245656691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4806e230 con 0x7f7d480cff70 2024-09-10T20:59:16.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.761+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/2245656691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d4806e7f0 con 0x7f7d480cff70 2024-09-10T20:59:16.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.762+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/2245656691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d480b0630 con 0x7f7d480cff70 2024-09-10T20:59:16.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.762+0000 7f7d45ffb640 1 -- 172.21.15.98:0/2245656691 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7d38005640 con 0x7f7d480cff70 2024-09-10T20:59:16.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.763+0000 7f7d45ffb640 1 --2- 172.21.15.98:0/2245656691 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7d34077f70 0x7f7d3407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:16.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.763+0000 7f7d45ffb640 1 -- 172.21.15.98:0/2245656691 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7d38099a00 con 0x7f7d480cff70 2024-09-10T20:59:16.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.763+0000 7f7d4cf3e640 1 --2- 172.21.15.98:0/2245656691 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7d34077f70 0x7f7d3407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:16.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.764+0000 7f7d4cf3e640 1 --2- 172.21.15.98:0/2245656691 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7d34077f70 0x7f7d3407a430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c034000 tx=0x7f7d3c0046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:16.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.767+0000 7f7d45ffb640 1 -- 172.21.15.98:0/2245656691 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7d480c6e80 con 0x7f7d480cff70 2024-09-10T20:59:16.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.978+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/2245656691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7d480d2b90 con 0x7f7d480cff70 2024-09-10T20:59:16.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.978+0000 7f7d45ffb640 1 -- 172.21.15.98:0/2245656691 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f7d380621c0 con 0x7f7d480cff70 2024-09-10T20:59:16.982 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T20:59:16.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.980+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/2245656691 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7d34077f70 msgr2=0x7f7d3407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:16.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.980+0000 7f7d4f9a3640 1 --2- 172.21.15.98:0/2245656691 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7d34077f70 0x7f7d3407a430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c034000 tx=0x7f7d3c0046c0 comp rx=0 tx=0).stop 2024-09-10T20:59:16.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.980+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/2245656691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d480cff70 msgr2=0x7f7d481502a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:16.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.980+0000 7f7d4f9a3640 1 --2- 172.21.15.98:0/2245656691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d480cff70 0x7f7d481502a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800c7a0 tx=0x7f7d3800cc60 comp rx=0 tx=0).stop 2024-09-10T20:59:16.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.980+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/2245656691 shutdown_connections 2024-09-10T20:59:16.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.980+0000 7f7d4f9a3640 1 --2- 172.21.15.98:0/2245656691 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7d34077f70 0x7f7d3407a430 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:16.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.980+0000 7f7d4f9a3640 1 --2- 172.21.15.98:0/2245656691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d4815c720 0x7f7d481e9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:16.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.980+0000 7f7d4f9a3640 1 --2- 172.21.15.98:0/2245656691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d48160530 0x7f7d481f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:16.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.980+0000 7f7d4f9a3640 1 --2- 172.21.15.98:0/2245656691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d480cff70 0x7f7d481502a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:16.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.980+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/2245656691 >> 172.21.15.98:0/2245656691 conn(0x7f7d480b37c0 msgr2=0x7f7d480af590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:16.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.980+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/2245656691 shutdown_connections 2024-09-10T20:59:16.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:16.980+0000 7f7d4f9a3640 1 -- 172.21.15.98:0/2245656691 wait complete. 2024-09-10T20:59:16.995 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T20:59:16.995 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T20:59:16.995 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-10T20:59:16.995 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-10T20:59:16.995 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T20:59:16.996 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T20:59:16.996 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T20:59:16.996 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T20:59:16.996 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-10T20:59:16.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:16.996 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-10T20:59:17.012 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T20:59:17.012 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T20:59:17.012 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T20:59:17.012 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:17.012 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-10T20:59:17.012 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:17.012 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:17.012 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:17.012 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-10T20:59:17.012 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:17.013 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:17.013 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-10T20:59:17.013 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:17.013 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:17.013 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T20:59:17.013 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:17.013 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T20:59:17.013 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42767sec preferred_lft 42767sec 2024-09-10T20:59:17.013 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link noprefixroute 2024-09-10T20:59:17.013 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:17.013 INFO:teuthology.orchestra.run.smithi098.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T20:59:17.013 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0a:5a:1d:cc:3c:77 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:17.013 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T20:59:17.014 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:17.014 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::e069:b2ff:feb8:4916/64 scope link 2024-09-10T20:59:17.014 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:17.014 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T20:59:17.014 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0a:5a:1d:cc:3c:77 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T20:59:17.014 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::85a:1dff:fecc:3c77/64 scope link tentative 2024-09-10T20:59:17.014 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:17.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:17.014 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T20:59:17.075 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-10T20:59:17.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:17.077 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-10T20:59:17.136 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-10T20:59:17.136 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T20:59:17.136 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T20:59:17.136 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T20:59:17.136 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T20:59:17.137 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:17.137 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:59:17.137 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T20:59:17.137 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T20:59:17.137 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T20:59:17.137 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:59:17.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:17.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:17.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:17.261 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-10T20:59:17.277 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T20:59:17.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:17.277 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list-id 2024-09-10T20:59:17.332 INFO:teuthology.orchestra.run.smithi098.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-10T20:59:17.333 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:17.333 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:59:17.333 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T20:59:17.334 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-10T20:59:17.334 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:59:17.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:17.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:17.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:17.449 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T20:59:17.474 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T20:59:17.474 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T20:59:17.474 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T20:59:17.474 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:17.474 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-10T20:59:17.474 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:17.474 INFO:teuthology.orchestra.run.smithi098.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T20:59:17.474 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether d6:93:81:69:53:47 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T20:59:17.475 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T20:59:17.475 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:17.475 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::d493:81ff:fe69:5347/64 scope link tentative 2024-09-10T20:59:17.475 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:17.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:17.476 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T20:59:17.540 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T20:59:17.540 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T20:59:17.540 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T20:59:17.540 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:17.541 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-10T20:59:17.541 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:17.541 INFO:teuthology.orchestra.run.smithi098.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T20:59:17.541 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether d6:93:81:69:53:47 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T20:59:17.541 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T20:59:17.541 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:17.541 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::d493:81ff:fe69:5347/64 scope link tentative 2024-09-10T20:59:17.541 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:17.542 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-10T20:59:17.543 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:17.543 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:59:17.543 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-10T20:59:17.543 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T20:59:17.543 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-10T20:59:17.543 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-10T20:59:17.543 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-10T20:59:17.543 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:59:17.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:17.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:17.694 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:17.694 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:59:17.694 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.1 up 2024-09-10T20:59:17.694 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T20:59:17.694 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:59:17.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:17.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:17.770 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T20:59:17.770 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-10T20:59:17.785 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-10T20:59:17.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T20:59:17.786 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-10T20:59:17.841 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-09-10T20:59:17.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:17.841 DEBUG:teuthology.orchestra.run.smithi098:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:17.924 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:17.924 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T20:59:17.931 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:17.931 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T20:59:17.931 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-09-10T20:59:17.931 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: trying mount with new device syntax: 2@41041377-a016-49fd-be1f-323d0b834557.cephfs=/ 2024-09-10T20:59:17.931 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.98:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2024-09-10T20:59:17.932 INFO:teuthology.orchestra.run.smithi098.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-10T20:59:17.932 INFO:teuthology.orchestra.run.smithi098.stdout: You just mounted a file system that supports labels which does not 2024-09-10T20:59:17.932 INFO:teuthology.orchestra.run.smithi098.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T20:59:17.932 INFO:teuthology.orchestra.run.smithi098.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T20:59:17.932 INFO:teuthology.orchestra.run.smithi098.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T20:59:17.934 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T20:59:17.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:17.934 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-10T20:59:17.998 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T20:59:18.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T20:59:18.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:18.094 INFO:teuthology.orchestra.run.smithi098.stdout:client4830 2024-09-10T20:59:18.094 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:59:18.094 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:18.110 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T20:59:18.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T20:59:18.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:18.206 INFO:teuthology.orchestra.run.smithi098.stdout:client4830 2024-09-10T20:59:18.206 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T20:59:18.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T20:59:18.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:18.262 INFO:teuthology.orchestra.run.smithi098.stdout:client4830 2024-09-10T20:59:18.262 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/41041377-a016-49fd-be1f-323d0b834557.client4830/status') 2024-09-10T20:59:18.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:18.320 INFO:teuthology.orchestra.run.smithi098.stdout:instance: client.4830 (3)192.168.0.2:0/489990435 2024-09-10T20:59:18.320 INFO:teuthology.orchestra.run.smithi098.stdout:blocklisted: false 2024-09-10T20:59:18.320 INFO:teuthology.orchestra.run.smithi098.stderr:0+1 records in 2024-09-10T20:59:18.320 INFO:teuthology.orchestra.run.smithi098.stderr:0+1 records out 2024-09-10T20:59:18.320 INFO:teuthology.orchestra.run.smithi098.stderr:68 bytes copied, 3.6833e-05 s, 1.8 MB/s 2024-09-10T20:59:18.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:18.328 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-09-10T20:59:18.328 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T20:59:18.328 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T20:59:18.328 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T20:59:18.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.426+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1687457449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b0073980 msgr2=0x7f62b0073df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:18.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.426+0000 7f62b5b9f640 1 --2- 172.21.15.98:0/1687457449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b0073980 0x7f62b0073df0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f62980098e0 tx=0x7f629802a530 comp rx=0 tx=0).stop 2024-09-10T20:59:18.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.426+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1687457449 shutdown_connections 2024-09-10T20:59:18.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.426+0000 7f62b5b9f640 1 --2- 172.21.15.98:0/1687457449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b0073980 0x7f62b0073df0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:18.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.426+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1687457449 >> 172.21.15.98:0/1687457449 conn(0x7f62b006ce40 msgr2=0x7f62b006d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:18.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.426+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1687457449 shutdown_connections 2024-09-10T20:59:18.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.426+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1687457449 wait complete. 2024-09-10T20:59:18.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.427+0000 7f62b5b9f640 1 Processor -- start 2024-09-10T20:59:18.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.427+0000 7f62b5b9f640 1 -- start start 2024-09-10T20:59:18.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b5b9f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62b006eea0 0x7f62b00ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:18.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b5b9f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62b0073980 0x7f62b013bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:18.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b5b9f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b015a850 0x7f62b00b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:18.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b5b9f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f62b006e510 con 0x7f62b015a850 2024-09-10T20:59:18.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b5b9f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f62b00b2c00 con 0x7f62b006eea0 2024-09-10T20:59:18.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b5b9f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f62b006e690 con 0x7f62b0073980 2024-09-10T20:59:18.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b539e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b015a850 0x7f62b00b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:18.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b539e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b015a850 0x7f62b00b1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55618/0 (socket says 172.21.15.98:55618) 2024-09-10T20:59:18.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b539e640 1 -- 172.21.15.98:0/1854522519 learned_addr learned my addr 172.21.15.98:0/1854522519 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:18.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b4b9d640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62b006eea0 0x7f62b00ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:18.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62affff640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62b0073980 0x7f62b013bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:18.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b539e640 1 -- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62b0073980 msgr2=0x7f62b013bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:18.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b539e640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62b0073980 0x7f62b013bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:18.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b539e640 1 -- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62b006eea0 msgr2=0x7f62b00ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:18.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b539e640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62b006eea0 0x7f62b00ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:18.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b539e640 1 -- 172.21.15.98:0/1854522519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b0156470 con 0x7f62b015a850 2024-09-10T20:59:18.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b4b9d640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62b006eea0 0x7f62b00ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:18.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.428+0000 7f62b539e640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b015a850 0x7f62b00b1730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f62a800c390 tx=0x7f62a800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:18.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.429+0000 7f62ad7fa640 1 -- 172.21.15.98:0/1854522519 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8017070 con 0x7f62b015a850 2024-09-10T20:59:18.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.429+0000 7f62ad7fa640 1 -- 172.21.15.98:0/1854522519 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62a80035f0 con 0x7f62b015a850 2024-09-10T20:59:18.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.429+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1854522519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b0143060 con 0x7f62b015a850 2024-09-10T20:59:18.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.429+0000 7f62ad7fa640 1 -- 172.21.15.98:0/1854522519 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8003a40 con 0x7f62b015a850 2024-09-10T20:59:18.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.429+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1854522519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b0140190 con 0x7f62b015a850 2024-09-10T20:59:18.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.430+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1854522519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b0156640 con 0x7f62b015a850 2024-09-10T20:59:18.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.430+0000 7f62ad7fa640 1 -- 172.21.15.98:0/1854522519 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f62a8003be0 con 0x7f62b015a850 2024-09-10T20:59:18.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.431+0000 7f62ad7fa640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6290077f70 0x7f629007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:18.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.431+0000 7f62ad7fa640 1 -- 172.21.15.98:0/1854522519 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f62a8099740 con 0x7f62b015a850 2024-09-10T20:59:18.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.431+0000 7f62b4b9d640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6290077f70 0x7f629007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:18.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.432+0000 7f62b4b9d640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6290077f70 0x7f629007a430 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f62a0001d10 tx=0x7f62a0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:18.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.435+0000 7f62ad7fa640 1 -- 172.21.15.98:0/1854522519 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f62a8061f00 con 0x7f62b015a850 2024-09-10T20:59:18.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.645+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1854522519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f62b0073df0 con 0x7f62b015a850 2024-09-10T20:59:18.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.645+0000 7f62ad7fa640 1 -- 172.21.15.98:0/1854522519 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f62a8061800 con 0x7f62b015a850 2024-09-10T20:59:18.650 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T20:59:18.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.647+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6290077f70 msgr2=0x7f629007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:18.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.647+0000 7f62b5b9f640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6290077f70 0x7f629007a430 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f62a0001d10 tx=0x7f62a0009040 comp rx=0 tx=0).stop 2024-09-10T20:59:18.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.647+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1854522519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b015a850 msgr2=0x7f62b00b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:18.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.647+0000 7f62b5b9f640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b015a850 0x7f62b00b1730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f62a800c390 tx=0x7f62a800c850 comp rx=0 tx=0).stop 2024-09-10T20:59:18.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.647+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1854522519 shutdown_connections 2024-09-10T20:59:18.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.647+0000 7f62b5b9f640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6290077f70 0x7f629007a430 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:18.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.647+0000 7f62b5b9f640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b015a850 0x7f62b00b1730 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:18.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.647+0000 7f62b5b9f640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62b0073980 0x7f62b013bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:18.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.647+0000 7f62b5b9f640 1 --2- 172.21.15.98:0/1854522519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62b006eea0 0x7f62b00ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:18.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.647+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1854522519 >> 172.21.15.98:0/1854522519 conn(0x7f62b006ce40 msgr2=0x7f62b006d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:18.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.648+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1854522519 shutdown_connections 2024-09-10T20:59:18.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:18.648+0000 7f62b5b9f640 1 -- 172.21.15.98:0/1854522519 wait complete. 2024-09-10T20:59:18.662 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T20:59:18.663 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T20:59:18.663 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2024-09-10T20:59:18.663 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T20:59:18.663 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T20:59:18.663 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T20:59:18.663 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T20:59:18.663 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T20:59:18.663 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T20:59:18.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:18.663 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2024-09-10T20:59:18.683 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T20:59:18.683 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T20:59:18.683 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T20:59:18.683 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:18.683 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-09-10T20:59:18.683 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:18.683 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:18.683 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:18.683 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2024-09-10T20:59:18.683 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:18.684 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:18.684 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2024-09-10T20:59:18.684 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:18.684 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:18.684 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T20:59:18.684 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:18.684 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T20:59:18.684 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42763sec preferred_lft 42763sec 2024-09-10T20:59:18.684 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link noprefixroute 2024-09-10T20:59:18.684 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:18.684 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T20:59:18.685 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:18.685 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-09-10T20:59:18.685 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add name ceph-brx type bridge 2024-09-10T20:59:18.685 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr flush dev ceph-brx 2024-09-10T20:59:18.685 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set ceph-brx up 2024-09-10T20:59:18.685 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T20:59:18.685 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-09-10T20:59:18.778 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:18.972 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:18.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:18.980 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T20:59:19.005 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-09-10T20:59:19.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:19.006 DEBUG:teuthology.orchestra.run.smithi190:> route 2024-09-10T20:59:19.080 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2024-09-10T20:59:19.080 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T20:59:19.080 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T20:59:19.080 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T20:59:19.080 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T20:59:19.080 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:19.081 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-09-10T20:59:19.081 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T20:59:19.081 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T20:59:19.081 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T20:59:19.081 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-09-10T20:59:19.167 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:19.424 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:19.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:19.430 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-09-10T20:59:19.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:19.446 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2024-09-10T20:59:19.502 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:19.502 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-09-10T20:59:19.502 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T20:59:19.502 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-10T20:59:19.502 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-09-10T20:59:19.591 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:19.617 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:19.623 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-10T20:59:19.624 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:19.624 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-09-10T20:59:19.624 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-10T20:59:19.624 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T20:59:19.624 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-10T20:59:19.624 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-10T20:59:19.624 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-10T20:59:19.625 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-09-10T20:59:19.674 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:19.761 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:19.767 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:19.767 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-09-10T20:59:19.768 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.0 up 2024-09-10T20:59:19.768 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T20:59:19.768 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-09-10T20:59:19.815 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:19.838 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:19.845 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T20:59:19.845 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T20:59:19.860 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T20:59:19.861 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T20:59:19.861 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T20:59:19.920 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T20:59:19.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:19.920 DEBUG:teuthology.orchestra.run.smithi190:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:20.012 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:20.012 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T20:59:20.149 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:20.150 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T20:59:20.150 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-09-10T20:59:20.150 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: trying mount with new device syntax: 1@41041377-a016-49fd-be1f-323d0b834557.cephfs=/ 2024-09-10T20:59:20.150 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.98:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2024-09-10T20:59:20.150 INFO:teuthology.orchestra.run.smithi190.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-10T20:59:20.150 INFO:teuthology.orchestra.run.smithi190.stdout: You just mounted a file system that supports labels which does not 2024-09-10T20:59:20.151 INFO:teuthology.orchestra.run.smithi190.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T20:59:20.151 INFO:teuthology.orchestra.run.smithi190.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T20:59:20.151 INFO:teuthology.orchestra.run.smithi190.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T20:59:20.152 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T20:59:20.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:20.153 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T20:59:20.177 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T20:59:20.266 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T20:59:20.268 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:20.274 INFO:teuthology.orchestra.run.smithi190.stdout:client4842 2024-09-10T20:59:20.274 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:59:20.274 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:20.290 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T20:59:20.379 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T20:59:20.380 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:20.386 INFO:teuthology.orchestra.run.smithi190.stdout:client4842 2024-09-10T20:59:20.386 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T20:59:20.434 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T20:59:20.436 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:20.442 INFO:teuthology.orchestra.run.smithi190.stdout:client4842 2024-09-10T20:59:20.442 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/41041377-a016-49fd-be1f-323d0b834557.client4842/status') 2024-09-10T20:59:20.490 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:20.500 INFO:teuthology.orchestra.run.smithi190.stdout:instance: client.4842 (3)172.21.15.190:0/3117542267 2024-09-10T20:59:20.501 INFO:teuthology.orchestra.run.smithi190.stdout:blocklisted: false 2024-09-10T20:59:20.501 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records in 2024-09-10T20:59:20.501 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records out 2024-09-10T20:59:20.501 INFO:teuthology.orchestra.run.smithi190.stderr:71 bytes copied, 2.2867e-05 s, 3.1 MB/s 2024-09-10T20:59:20.501 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:20.508 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-09-10T20:59:20.508 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T20:59:20.508 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T20:59:20.508 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T20:59:20.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.617+0000 7f38afd8a640 1 -- 172.21.15.98:0/395823144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a806fe60 msgr2=0x7f38a815df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:20.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.617+0000 7f38afd8a640 1 --2- 172.21.15.98:0/395823144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a806fe60 0x7f38a815df70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f38a0005fd0 tx=0x7f38a002aa10 comp rx=0 tx=0).stop 2024-09-10T20:59:20.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.617+0000 7f38afd8a640 1 -- 172.21.15.98:0/395823144 shutdown_connections 2024-09-10T20:59:20.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.617+0000 7f38afd8a640 1 --2- 172.21.15.98:0/395823144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38a815b1e0 0x7f38a815da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:20.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.617+0000 7f38afd8a640 1 --2- 172.21.15.98:0/395823144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a806fe60 0x7f38a815df70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:20.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.617+0000 7f38afd8a640 1 -- 172.21.15.98:0/395823144 >> 172.21.15.98:0/395823144 conn(0x7f38a80b1760 msgr2=0x7f38a80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:20.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.617+0000 7f38afd8a640 1 -- 172.21.15.98:0/395823144 shutdown_connections 2024-09-10T20:59:20.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.617+0000 7f38afd8a640 1 -- 172.21.15.98:0/395823144 wait complete. 2024-09-10T20:59:20.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.618+0000 7f38afd8a640 1 Processor -- start 2024-09-10T20:59:20.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.618+0000 7f38afd8a640 1 -- start start 2024-09-10T20:59:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38afd8a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a806fe60 0x7f38a814e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38afd8a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38a813cf80 0x7f38a813a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38afd8a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38a815b1e0 0x7f38a814cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38afd8a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8164f40 con 0x7f38a806fe60 2024-09-10T20:59:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38afd8a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8164dc0 con 0x7f38a813cf80 2024-09-10T20:59:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38afd8a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f38a81650c0 con 0x7f38a815b1e0 2024-09-10T20:59:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38adb26640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a806fe60 0x7f38a814e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38adb26640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a806fe60 0x7f38a814e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39288/0 (socket says 172.21.15.98:39288) 2024-09-10T20:59:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38adb26640 1 -- 172.21.15.98:0/1071091221 learned_addr learned my addr 172.21.15.98:0/1071091221 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38ad325640 1 --2- 172.21.15.98:0/1071091221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38a813cf80 0x7f38a813a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38ae327640 1 --2- 172.21.15.98:0/1071091221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38a815b1e0 0x7f38a814cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38adb26640 1 -- 172.21.15.98:0/1071091221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38a815b1e0 msgr2=0x7f38a814cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38adb26640 1 --2- 172.21.15.98:0/1071091221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38a815b1e0 0x7f38a814cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38adb26640 1 -- 172.21.15.98:0/1071091221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38a813cf80 msgr2=0x7f38a813a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38adb26640 1 --2- 172.21.15.98:0/1071091221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38a813cf80 0x7f38a813a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.619+0000 7f38adb26640 1 -- 172.21.15.98:0/1071091221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a81398d0 con 0x7f38a806fe60 2024-09-10T20:59:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.620+0000 7f38adb26640 1 --2- 172.21.15.98:0/1071091221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a806fe60 0x7f38a814e0d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f38a400c450 tx=0x7f38a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.620+0000 7f389e7fc640 1 -- 172.21.15.98:0/1071091221 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a40052d0 con 0x7f38a806fe60 2024-09-10T20:59:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.620+0000 7f389e7fc640 1 -- 172.21.15.98:0/1071091221 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38a4005470 con 0x7f38a806fe60 2024-09-10T20:59:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.620+0000 7f38afd8a640 1 -- 172.21.15.98:0/1071091221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a8139bb0 con 0x7f38a806fe60 2024-09-10T20:59:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.620+0000 7f389e7fc640 1 -- 172.21.15.98:0/1071091221 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4012870 con 0x7f38a806fe60 2024-09-10T20:59:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.620+0000 7f38afd8a640 1 -- 172.21.15.98:0/1071091221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a813a170 con 0x7f38a806fe60 2024-09-10T20:59:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.621+0000 7f38afd8a640 1 -- 172.21.15.98:0/1071091221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a806f750 con 0x7f38a806fe60 2024-09-10T20:59:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.621+0000 7f389e7fc640 1 -- 172.21.15.98:0/1071091221 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f38a40035f0 con 0x7f38a806fe60 2024-09-10T20:59:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.622+0000 7f389e7fc640 1 --2- 172.21.15.98:0/1071091221 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f388c077fa0 0x7f388c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.622+0000 7f389e7fc640 1 -- 172.21.15.98:0/1071091221 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f38a4099a70 con 0x7f38a806fe60 2024-09-10T20:59:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.622+0000 7f38ad325640 1 --2- 172.21.15.98:0/1071091221 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f388c077fa0 0x7f388c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.622+0000 7f38ad325640 1 --2- 172.21.15.98:0/1071091221 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f388c077fa0 0x7f388c07a460 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f38a814e940 tx=0x7f389800b2e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:20.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.626+0000 7f389e7fc640 1 -- 172.21.15.98:0/1071091221 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f38a4061be0 con 0x7f38a806fe60 2024-09-10T20:59:20.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.837+0000 7f38afd8a640 1 -- 172.21.15.98:0/1071091221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f38a80cd660 con 0x7f38a806fe60 2024-09-10T20:59:20.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.838+0000 7f389e7fc640 1 -- 172.21.15.98:0/1071091221 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f38a4066610 con 0x7f38a806fe60 2024-09-10T20:59:20.843 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T20:59:20.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.840+0000 7f38afd8a640 1 -- 172.21.15.98:0/1071091221 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f388c077fa0 msgr2=0x7f388c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:20.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.840+0000 7f38afd8a640 1 --2- 172.21.15.98:0/1071091221 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f388c077fa0 0x7f388c07a460 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f38a814e940 tx=0x7f389800b2e0 comp rx=0 tx=0).stop 2024-09-10T20:59:20.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.840+0000 7f38afd8a640 1 -- 172.21.15.98:0/1071091221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a806fe60 msgr2=0x7f38a814e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:20.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.840+0000 7f38afd8a640 1 --2- 172.21.15.98:0/1071091221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a806fe60 0x7f38a814e0d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f38a400c450 tx=0x7f38a400c910 comp rx=0 tx=0).stop 2024-09-10T20:59:20.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.840+0000 7f38afd8a640 1 -- 172.21.15.98:0/1071091221 shutdown_connections 2024-09-10T20:59:20.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.840+0000 7f38afd8a640 1 --2- 172.21.15.98:0/1071091221 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f388c077fa0 0x7f388c07a460 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:20.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.840+0000 7f38afd8a640 1 --2- 172.21.15.98:0/1071091221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38a815b1e0 0x7f38a814cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:20.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.840+0000 7f38afd8a640 1 --2- 172.21.15.98:0/1071091221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38a813cf80 0x7f38a813a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:20.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.840+0000 7f38afd8a640 1 --2- 172.21.15.98:0/1071091221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38a806fe60 0x7f38a814e0d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:20.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.840+0000 7f38afd8a640 1 -- 172.21.15.98:0/1071091221 >> 172.21.15.98:0/1071091221 conn(0x7f38a80b1760 msgr2=0x7f38a814e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:20.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.840+0000 7f38afd8a640 1 -- 172.21.15.98:0/1071091221 shutdown_connections 2024-09-10T20:59:20.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:20.840+0000 7f38afd8a640 1 -- 172.21.15.98:0/1071091221 wait complete. 2024-09-10T20:59:20.855 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T20:59:20.855 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T20:59:20.855 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2024-09-10T20:59:20.855 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-10T20:59:20.855 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T20:59:20.855 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T20:59:20.855 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T20:59:20.855 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T20:59:20.855 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-10T20:59:20.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:20.856 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2024-09-10T20:59:20.872 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T20:59:20.872 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T20:59:20.872 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T20:59:20.872 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:20.872 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-09-10T20:59:20.873 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:20.873 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:20.873 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:20.873 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2024-09-10T20:59:20.873 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:20.873 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:20.873 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2024-09-10T20:59:20.873 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:20.873 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:20.873 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T20:59:20.873 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:20.874 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T20:59:20.874 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42761sec preferred_lft 42761sec 2024-09-10T20:59:20.874 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link noprefixroute 2024-09-10T20:59:20.874 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:20.874 INFO:teuthology.orchestra.run.smithi190.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T20:59:20.874 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether ca:f2:36:11:ce:0e brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:20.874 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T20:59:20.874 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:20.874 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::94eb:25ff:fe86:2433/64 scope link 2024-09-10T20:59:20.874 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:20.874 INFO:teuthology.orchestra.run.smithi190.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T20:59:20.874 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether ca:f2:36:11:ce:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T20:59:20.875 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::c8f2:36ff:fe11:ce0e/64 scope link tentative 2024-09-10T20:59:20.875 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:20.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:20.875 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T20:59:20.937 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-09-10T20:59:20.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:20.938 DEBUG:teuthology.orchestra.run.smithi190:> route 2024-09-10T20:59:20.995 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2024-09-10T20:59:20.996 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T20:59:20.996 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T20:59:20.996 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T20:59:20.997 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T20:59:20.997 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:20.997 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-09-10T20:59:20.998 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T20:59:20.998 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T20:59:20.998 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T20:59:20.998 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-09-10T20:59:21.083 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:21.115 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:21.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:21.121 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-09-10T20:59:21.137 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T20:59:21.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:21.138 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2024-09-10T20:59:21.193 INFO:teuthology.orchestra.run.smithi190.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-10T20:59:21.194 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:21.194 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-09-10T20:59:21.194 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T20:59:21.194 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-10T20:59:21.194 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-09-10T20:59:21.282 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:21.305 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:21.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:21.310 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T20:59:21.335 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T20:59:21.335 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T20:59:21.335 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T20:59:21.335 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:21.335 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-09-10T20:59:21.335 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:21.335 INFO:teuthology.orchestra.run.smithi190.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T20:59:21.335 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether be:6b:85:8c:a9:6e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T20:59:21.335 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T20:59:21.335 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:21.336 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::bc6b:85ff:fe8c:a96e/64 scope link tentative 2024-09-10T20:59:21.336 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:21.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:21.337 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T20:59:21.401 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T20:59:21.402 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T20:59:21.402 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T20:59:21.402 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:21.402 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-09-10T20:59:21.402 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:21.402 INFO:teuthology.orchestra.run.smithi190.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T20:59:21.402 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether be:6b:85:8c:a9:6e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T20:59:21.402 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T20:59:21.402 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:21.402 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::bc6b:85ff:fe8c:a96e/64 scope link tentative 2024-09-10T20:59:21.402 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:21.403 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-10T20:59:21.404 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:21.404 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-09-10T20:59:21.404 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-10T20:59:21.404 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T20:59:21.404 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-10T20:59:21.404 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-10T20:59:21.404 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-10T20:59:21.404 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-09-10T20:59:21.493 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:21.550 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:21.556 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:21.557 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-09-10T20:59:21.557 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.1 up 2024-09-10T20:59:21.557 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T20:59:21.557 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-09-10T20:59:21.605 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:21.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:21.633 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T20:59:21.633 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-10T20:59:21.647 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-10T20:59:21.648 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T20:59:21.648 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-10T20:59:21.703 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-09-10T20:59:21.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:21.703 DEBUG:teuthology.orchestra.run.smithi190:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:21.786 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:21.787 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T20:59:21.794 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:21.795 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T20:59:21.795 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-09-10T20:59:21.795 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: trying mount with new device syntax: 3@41041377-a016-49fd-be1f-323d0b834557.cephfs=/ 2024-09-10T20:59:21.795 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.98:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2024-09-10T20:59:21.796 INFO:teuthology.orchestra.run.smithi190.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-10T20:59:21.796 INFO:teuthology.orchestra.run.smithi190.stdout: You just mounted a file system that supports labels which does not 2024-09-10T20:59:21.796 INFO:teuthology.orchestra.run.smithi190.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T20:59:21.796 INFO:teuthology.orchestra.run.smithi190.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T20:59:21.796 INFO:teuthology.orchestra.run.smithi190.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T20:59:21.797 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T20:59:21.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:21.798 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-10T20:59:21.822 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T20:59:21.911 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T20:59:21.912 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:21.918 INFO:teuthology.orchestra.run.smithi190.stdout:client4535 2024-09-10T20:59:21.918 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:59:21.918 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:21.934 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T20:59:22.021 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T20:59:22.023 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:22.029 INFO:teuthology.orchestra.run.smithi190.stdout:client4535 2024-09-10T20:59:22.029 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T20:59:22.077 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T20:59:22.079 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:22.085 INFO:teuthology.orchestra.run.smithi190.stdout:client4535 2024-09-10T20:59:22.085 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/41041377-a016-49fd-be1f-323d0b834557.client4535/status') 2024-09-10T20:59:22.133 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:22.143 INFO:teuthology.orchestra.run.smithi190.stdout:instance: client.4535 (3)192.168.0.2:0/586945935 2024-09-10T20:59:22.143 INFO:teuthology.orchestra.run.smithi190.stdout:blocklisted: false 2024-09-10T20:59:22.143 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records in 2024-09-10T20:59:22.143 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records out 2024-09-10T20:59:22.144 INFO:teuthology.orchestra.run.smithi190.stderr:68 bytes copied, 2.2198e-05 s, 3.1 MB/s 2024-09-10T20:59:22.144 INFO:teuthology.orchestra.run.smithi190.stderr:2024-09-10T20:59:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:22.150 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-10T20:59:22.174 INFO:tasks.cephfs_test_runner:Starting test: test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush) MDS starts to handle client caps when it enters clientreplay stage. 2024-09-10T20:59:22.175 INFO:tasks.cephfs_test_runner:test_replay_create (tasks.cephfs.test_cap_flush.TestCapFlush) 2024-09-10T20:59:22.175 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:59:22.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.283+0000 7f79e74f6640 1 -- 172.21.15.98:0/2788325284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e00d2a20 msgr2=0x7f79e00cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:22.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.283+0000 7f79e74f6640 1 --2- 172.21.15.98:0/2788325284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e00d2a20 0x7f79e00cea40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f79d8005fd0 tx=0x7f79d802aa90 comp rx=0 tx=0).stop 2024-09-10T20:59:22.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.283+0000 7f79e74f6640 1 -- 172.21.15.98:0/2788325284 shutdown_connections 2024-09-10T20:59:22.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.283+0000 7f79e74f6640 1 --2- 172.21.15.98:0/2788325284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79e00d4af0 0x7f79e00d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.283+0000 7f79e74f6640 1 --2- 172.21.15.98:0/2788325284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e00d2a20 0x7f79e00cea40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.283+0000 7f79e74f6640 1 -- 172.21.15.98:0/2788325284 >> 172.21.15.98:0/2788325284 conn(0x7f79e00d3f70 msgr2=0x7f79e00d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:22.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.284+0000 7f79e74f6640 1 -- 172.21.15.98:0/2788325284 shutdown_connections 2024-09-10T20:59:22.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.284+0000 7f79e74f6640 1 -- 172.21.15.98:0/2788325284 wait complete. 2024-09-10T20:59:22.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.285+0000 7f79e74f6640 1 Processor -- start 2024-09-10T20:59:22.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.285+0000 7f79e74f6640 1 -- start start 2024-09-10T20:59:22.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.285+0000 7f79e74f6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79e00d2a20 0x7f79e0154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:22.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e74f6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e00d4af0 0x7f79e00c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:22.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e74f6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79e015e060 0x7f79e00ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:22.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e74f6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f79e00cf100 con 0x7f79e00d4af0 2024-09-10T20:59:22.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e74f6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f79e00cef80 con 0x7f79e015e060 2024-09-10T20:59:22.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e74f6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f79e00cf280 con 0x7f79e00d2a20 2024-09-10T20:59:22.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e4a91640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e00d4af0 0x7f79e00c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:22.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e4a91640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e00d4af0 0x7f79e00c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39306/0 (socket says 172.21.15.98:39306) 2024-09-10T20:59:22.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e5292640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79e00d2a20 0x7f79e0154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:22.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e4a91640 1 -- 172.21.15.98:0/2817799786 learned_addr learned my addr 172.21.15.98:0/2817799786 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:22.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e5a93640 1 --2- 172.21.15.98:0/2817799786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79e015e060 0x7f79e00ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:22.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e4a91640 1 -- 172.21.15.98:0/2817799786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79e00d2a20 msgr2=0x7f79e0154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:22.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e4a91640 1 --2- 172.21.15.98:0/2817799786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79e00d2a20 0x7f79e0154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e4a91640 1 -- 172.21.15.98:0/2817799786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79e015e060 msgr2=0x7f79e00ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:22.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e4a91640 1 --2- 172.21.15.98:0/2817799786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79e015e060 0x7f79e00ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e4a91640 1 -- 172.21.15.98:0/2817799786 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e01f4ce0 con 0x7f79e00d4af0 2024-09-10T20:59:22.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e4a91640 1 --2- 172.21.15.98:0/2817799786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e00d4af0 0x7f79e00c9c10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00d980 tx=0x7f79dc00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:22.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79d5ffb640 1 -- 172.21.15.98:0/2817799786 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc017030 con 0x7f79e00d4af0 2024-09-10T20:59:22.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79d5ffb640 1 -- 172.21.15.98:0/2817799786 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79dc009b20 con 0x7f79e00d4af0 2024-09-10T20:59:22.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79e74f6640 1 -- 172.21.15.98:0/2817799786 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e01f4fc0 con 0x7f79e00d4af0 2024-09-10T20:59:22.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.286+0000 7f79d5ffb640 1 -- 172.21.15.98:0/2817799786 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc004020 con 0x7f79e00d4af0 2024-09-10T20:59:22.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.287+0000 7f79e74f6640 1 -- 172.21.15.98:0/2817799786 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e01f5580 con 0x7f79e00d4af0 2024-09-10T20:59:22.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.287+0000 7f79e74f6640 1 -- 172.21.15.98:0/2817799786 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e00d2840 con 0x7f79e00d4af0 2024-09-10T20:59:22.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.288+0000 7f79d5ffb640 1 -- 172.21.15.98:0/2817799786 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f79dc0041c0 con 0x7f79e00d4af0 2024-09-10T20:59:22.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.288+0000 7f79d5ffb640 1 --2- 172.21.15.98:0/2817799786 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f79c4077fa0 0x7f79c407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:22.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.289+0000 7f79e5292640 1 --2- 172.21.15.98:0/2817799786 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f79c4077fa0 0x7f79c407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:22.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.289+0000 7f79d5ffb640 1 -- 172.21.15.98:0/2817799786 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f79dc09a010 con 0x7f79e00d4af0 2024-09-10T20:59:22.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.289+0000 7f79e5292640 1 --2- 172.21.15.98:0/2817799786 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f79c4077fa0 0x7f79c407a460 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f79d0001720 tx=0x7f79d0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:22.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.293+0000 7f79d5ffb640 1 -- 172.21.15.98:0/2817799786 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f79e00d2840 con 0x7f79e00d4af0 2024-09-10T20:59:22.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.473+0000 7f79e74f6640 1 -- 172.21.15.98:0/2817799786 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79e00cfef0 con 0x7f79e00d4af0 2024-09-10T20:59:22.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.474+0000 7f79d5ffb640 1 -- 172.21.15.98:0/2817799786 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18610 (secure 0 0 0) 0x7f79dc062210 con 0x7f79e00d4af0 2024-09-10T20:59:22.478 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:22.478 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":25,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:58:51.602367+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:58:44.390290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T20:58:45.564099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T20:58:49.250508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:59:22.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.476+0000 7f79e74f6640 1 -- 172.21.15.98:0/2817799786 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f79c4077fa0 msgr2=0x7f79c407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:22.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.476+0000 7f79e74f6640 1 --2- 172.21.15.98:0/2817799786 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f79c4077fa0 0x7f79c407a460 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f79d0001720 tx=0x7f79d0009040 comp rx=0 tx=0).stop 2024-09-10T20:59:22.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.476+0000 7f79e74f6640 1 -- 172.21.15.98:0/2817799786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e00d4af0 msgr2=0x7f79e00c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.476+0000 7f79e74f6640 1 --2- 172.21.15.98:0/2817799786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e00d4af0 0x7f79e00c9c10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00d980 tx=0x7f79dc00de40 comp rx=0 tx=0).stop 2024-09-10T20:59:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.476+0000 7f79e74f6640 1 -- 172.21.15.98:0/2817799786 shutdown_connections 2024-09-10T20:59:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.476+0000 7f79e74f6640 1 --2- 172.21.15.98:0/2817799786 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f79c4077fa0 0x7f79c407a460 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.476+0000 7f79e74f6640 1 --2- 172.21.15.98:0/2817799786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79e015e060 0x7f79e00ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.476+0000 7f79e74f6640 1 --2- 172.21.15.98:0/2817799786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79e00d4af0 0x7f79e00c9c10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.476+0000 7f79e74f6640 1 --2- 172.21.15.98:0/2817799786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79e00d2a20 0x7f79e0154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.476+0000 7f79e74f6640 1 -- 172.21.15.98:0/2817799786 >> 172.21.15.98:0/2817799786 conn(0x7f79e00d3f70 msgr2=0x7f79e0162280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.476+0000 7f79e74f6640 1 -- 172.21.15.98:0/2817799786 shutdown_connections 2024-09-10T20:59:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.476+0000 7f79e74f6640 1 -- 172.21.15.98:0/2817799786 wait complete. 2024-09-10T20:59:22.491 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T20:58:34.920029+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T20:58:44.390290+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T20:58:45.564099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T20:58:49.250508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T20:59:22.492 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T20:59:22.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.592+0000 7fe50c457640 1 -- 172.21.15.98:0/3807807649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5040cf110 msgr2=0x7fe5040b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:22.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.592+0000 7fe50c457640 1 --2- 172.21.15.98:0/3807807649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5040cf110 0x7fe5040b47b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc005fd0 tx=0x7fe4fc02aa00 comp rx=0 tx=0).stop 2024-09-10T20:59:22.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.592+0000 7fe50c457640 1 -- 172.21.15.98:0/3807807649 shutdown_connections 2024-09-10T20:59:22.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.592+0000 7fe50c457640 1 --2- 172.21.15.98:0/3807807649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5040cf110 0x7fe5040b47b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.592+0000 7fe50c457640 1 --2- 172.21.15.98:0/3807807649 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe50415e150 0x7fe50415e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.592+0000 7fe50c457640 1 -- 172.21.15.98:0/3807807649 >> 172.21.15.98:0/3807807649 conn(0x7fe5040b1760 msgr2=0x7fe5040b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:22.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.592+0000 7fe50c457640 1 -- 172.21.15.98:0/3807807649 shutdown_connections 2024-09-10T20:59:22.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.592+0000 7fe50c457640 1 -- 172.21.15.98:0/3807807649 wait complete. 2024-09-10T20:59:22.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.593+0000 7fe50c457640 1 Processor -- start 2024-09-10T20:59:22.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.593+0000 7fe50c457640 1 -- start start 2024-09-10T20:59:22.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe50c457640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe50406e2e0 0x7fe5041515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:22.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe50c457640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5040cf110 0x7fe5041495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:22.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe50c457640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe50415e150 0x7fe5040c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:22.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe50c457640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe5040b3470 con 0x7fe5040cf110 2024-09-10T20:59:22.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe50c457640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe504072980 con 0x7fe50415e150 2024-09-10T20:59:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe50c457640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe5040b35f0 con 0x7fe50406e2e0 2024-09-10T20:59:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe5099f2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5040cf110 0x7fe5041495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe5099f2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5040cf110 0x7fe5041495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39316/0 (socket says 172.21.15.98:39316) 2024-09-10T20:59:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe5099f2640 1 -- 172.21.15.98:0/408279027 learned_addr learned my addr 172.21.15.98:0/408279027 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe50a1f3640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe50406e2e0 0x7fe5041515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe50a9f4640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe50415e150 0x7fe5040c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe5099f2640 1 -- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe50406e2e0 msgr2=0x7fe5041515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe5099f2640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe50406e2e0 0x7fe5041515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe5099f2640 1 -- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe50415e150 msgr2=0x7fe5040c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:22.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe5099f2640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe50415e150 0x7fe5040c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe5099f2640 1 -- 172.21.15.98:0/408279027 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5041e4090 con 0x7fe5040cf110 2024-09-10T20:59:22.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe50a9f4640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe50415e150 0x7fe5040c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:22.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe50a1f3640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe50406e2e0 0x7fe5041515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:22.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe5099f2640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5040cf110 0x7fe5041495b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400c430 tx=0x7fe4f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:22.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe4faffd640 1 -- 172.21.15.98:0/408279027 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4011790 con 0x7fe5040cf110 2024-09-10T20:59:22.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe4faffd640 1 -- 172.21.15.98:0/408279027 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4f4017070 con 0x7fe5040cf110 2024-09-10T20:59:22.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe50c457640 1 -- 172.21.15.98:0/408279027 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5041e43d0 con 0x7fe5040cf110 2024-09-10T20:59:22.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.594+0000 7fe4faffd640 1 -- 172.21.15.98:0/408279027 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4011cc0 con 0x7fe5040cf110 2024-09-10T20:59:22.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.595+0000 7fe50c457640 1 -- 172.21.15.98:0/408279027 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5041e4c10 con 0x7fe5040cf110 2024-09-10T20:59:22.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.595+0000 7fe50c457640 1 -- 172.21.15.98:0/408279027 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5040b47b0 con 0x7fe5040cf110 2024-09-10T20:59:22.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.596+0000 7fe4faffd640 1 -- 172.21.15.98:0/408279027 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe4f401c460 con 0x7fe5040cf110 2024-09-10T20:59:22.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.596+0000 7fe4faffd640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fe4e8077fa0 0x7fe4e807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:22.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.597+0000 7fe4faffd640 1 -- 172.21.15.98:0/408279027 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe4f4099ec0 con 0x7fe5040cf110 2024-09-10T20:59:22.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.597+0000 7fe50a1f3640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fe4e8077fa0 0x7fe4e807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:22.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.597+0000 7fe50a1f3640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fe4e8077fa0 0x7fe4e807a460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe500005fd0 tx=0x7fe500005e00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:22.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.601+0000 7fe4faffd640 1 -- 172.21.15.98:0/408279027 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe5041e43d0 con 0x7fe5040cf110 2024-09-10T20:59:22.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.773+0000 7fe50c457640 1 -- 172.21.15.98:0/408279027 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe5040d06e0 con 0x7fe5040cf110 2024-09-10T20:59:22.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.774+0000 7fe4faffd640 1 -- 172.21.15.98:0/408279027 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fe4f4062690 con 0x7fe5040cf110 2024-09-10T20:59:22.778 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-09-10T20:59:22.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.776+0000 7fe50c457640 1 -- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fe4e8077fa0 msgr2=0x7fe4e807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:22.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.776+0000 7fe50c457640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fe4e8077fa0 0x7fe4e807a460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe500005fd0 tx=0x7fe500005e00 comp rx=0 tx=0).stop 2024-09-10T20:59:22.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.776+0000 7fe50c457640 1 -- 172.21.15.98:0/408279027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5040cf110 msgr2=0x7fe5041495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:22.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.776+0000 7fe50c457640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5040cf110 0x7fe5041495b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400c430 tx=0x7fe4f400c8f0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.776+0000 7fe50c457640 1 -- 172.21.15.98:0/408279027 shutdown_connections 2024-09-10T20:59:22.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.776+0000 7fe50c457640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fe4e8077fa0 0x7fe4e807a460 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.776+0000 7fe50c457640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe50415e150 0x7fe5040c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.776+0000 7fe50c457640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5040cf110 0x7fe5041495b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.776+0000 7fe50c457640 1 --2- 172.21.15.98:0/408279027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe50406e2e0 0x7fe5041515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.776+0000 7fe50c457640 1 -- 172.21.15.98:0/408279027 >> 172.21.15.98:0/408279027 conn(0x7fe5040b1760 msgr2=0x7fe5040b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:22.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.776+0000 7fe50c457640 1 -- 172.21.15.98:0/408279027 shutdown_connections 2024-09-10T20:59:22.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.776+0000 7fe50c457640 1 -- 172.21.15.98:0/408279027 wait complete. 2024-09-10T20:59:22.791 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T20:59:22.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.891+0000 7fd7195c2640 1 -- 172.21.15.98:0/1450751200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd700004d40 msgr2=0x7fd7000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:22.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.891+0000 7fd7195c2640 1 --2- 172.21.15.98:0/1450751200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd700004d40 0x7fd7000051c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd70c001660 tx=0x7fd700005830 comp rx=0 tx=0).stop 2024-09-10T20:59:22.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.891+0000 7fd7137fe640 1 -- 172.21.15.98:0/1450751200 reap_dead start 2024-09-10T20:59:22.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.891+0000 7fd7195c2640 1 -- 172.21.15.98:0/1450751200 shutdown_connections 2024-09-10T20:59:22.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.891+0000 7fd7195c2640 1 -- 172.21.15.98:0/1450751200 >> 172.21.15.98:0/1450751200 conn(0x7fd7140d1d70 msgr2=0x7fd7140d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:22.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.891+0000 7fd7195c2640 1 -- 172.21.15.98:0/1450751200 shutdown_connections 2024-09-10T20:59:22.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.891+0000 7fd7195c2640 1 -- 172.21.15.98:0/1450751200 wait complete. 2024-09-10T20:59:22.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.892+0000 7fd7195c2640 1 Processor -- start 2024-09-10T20:59:22.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.892+0000 7fd7195c2640 1 -- start start 2024-09-10T20:59:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7195c2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd700004d40 0x7fd714156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7195c2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7141f6b00 0x7fd714156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7195c2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd714071050 0x7fd7140714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7195c2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd7140d0bc0 con 0x7fd7141f6b00 2024-09-10T20:59:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7195c2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd7140d0a40 con 0x7fd714071050 2024-09-10T20:59:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7195c2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd71415ef40 con 0x7fd700004d40 2024-09-10T20:59:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7127fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7141f6b00 0x7fd714156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7127fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7141f6b00 0x7fd714156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39342/0 (socket says 172.21.15.98:39342) 2024-09-10T20:59:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7127fc640 1 -- 172.21.15.98:0/2839739141 learned_addr learned my addr 172.21.15.98:0/2839739141 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd712ffd640 1 --2- 172.21.15.98:0/2839739141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd700004d40 0x7fd714156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7137fe640 1 --2- 172.21.15.98:0/2839739141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd714071050 0x7fd7140714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7127fc640 1 -- 172.21.15.98:0/2839739141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd700004d40 msgr2=0x7fd714156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7127fc640 1 --2- 172.21.15.98:0/2839739141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd700004d40 0x7fd714156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7127fc640 1 -- 172.21.15.98:0/2839739141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd714071050 msgr2=0x7fd7140714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7127fc640 1 --2- 172.21.15.98:0/2839739141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd714071050 0x7fd7140714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7127fc640 1 -- 172.21.15.98:0/2839739141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd714149e70 con 0x7fd7141f6b00 2024-09-10T20:59:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7127fc640 1 --2- 172.21.15.98:0/2839739141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7141f6b00 0x7fd714156b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc011b10 tx=0x7fd6fc009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd70b7fe640 1 -- 172.21.15.98:0/2839739141 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc01e070 con 0x7fd7141f6b00 2024-09-10T20:59:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd70b7fe640 1 -- 172.21.15.98:0/2839739141 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6fc004050 con 0x7fd7141f6b00 2024-09-10T20:59:22.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd70b7fe640 1 -- 172.21.15.98:0/2839739141 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc00fea0 con 0x7fd7141f6b00 2024-09-10T20:59:22.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.893+0000 7fd7195c2640 1 -- 172.21.15.98:0/2839739141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7141492f0 con 0x7fd7141f6b00 2024-09-10T20:59:22.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.894+0000 7fd7195c2640 1 -- 172.21.15.98:0/2839739141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7141498b0 con 0x7fd7141f6b00 2024-09-10T20:59:22.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.895+0000 7fd7195c2640 1 -- 172.21.15.98:0/2839739141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7140d0330 con 0x7fd7141f6b00 2024-09-10T20:59:22.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.895+0000 7fd70b7fe640 1 -- 172.21.15.98:0/2839739141 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd6fc0035f0 con 0x7fd7141f6b00 2024-09-10T20:59:22.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.896+0000 7fd70b7fe640 1 --2- 172.21.15.98:0/2839739141 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd6f4077fa0 0x7fd6f407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:22.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.896+0000 7fd712ffd640 1 --2- 172.21.15.98:0/2839739141 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd6f4077fa0 0x7fd6f407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:22.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.896+0000 7fd70b7fe640 1 -- 172.21.15.98:0/2839739141 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd6fc09ad80 con 0x7fd7141f6b00 2024-09-10T20:59:22.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.896+0000 7fd712ffd640 1 --2- 172.21.15.98:0/2839739141 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd6f4077fa0 0x7fd6f407a460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd704006fd0 tx=0x7fd704006b20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:22.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:22.900+0000 7fd70b7fe640 1 -- 172.21.15.98:0/2839739141 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd6fc062fb0 con 0x7fd7141f6b00 2024-09-10T20:59:23.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.074+0000 7fd7195c2640 1 -- 172.21.15.98:0/2839739141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd7140d0780 con 0x7fd7141f6b00 2024-09-10T20:59:23.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.074+0000 7fd70b7fe640 1 -- 172.21.15.98:0/2839739141 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fd6fc0679e0 con 0x7fd7141f6b00 2024-09-10T20:59:23.078 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-09-10T20:59:23.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.076+0000 7fd7195c2640 1 -- 172.21.15.98:0/2839739141 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd6f4077fa0 msgr2=0x7fd6f407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.076+0000 7fd7195c2640 1 --2- 172.21.15.98:0/2839739141 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd6f4077fa0 0x7fd6f407a460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd704006fd0 tx=0x7fd704006b20 comp rx=0 tx=0).stop 2024-09-10T20:59:23.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.076+0000 7fd7195c2640 1 -- 172.21.15.98:0/2839739141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7141f6b00 msgr2=0x7fd714156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.076+0000 7fd7195c2640 1 --2- 172.21.15.98:0/2839739141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7141f6b00 0x7fd714156b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc011b10 tx=0x7fd6fc009e90 comp rx=0 tx=0).stop 2024-09-10T20:59:23.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.076+0000 7fd7195c2640 1 -- 172.21.15.98:0/2839739141 shutdown_connections 2024-09-10T20:59:23.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.076+0000 7fd7195c2640 1 --2- 172.21.15.98:0/2839739141 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd6f4077fa0 0x7fd6f407a460 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.076+0000 7fd7195c2640 1 --2- 172.21.15.98:0/2839739141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd714071050 0x7fd7140714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.076+0000 7fd7195c2640 1 --2- 172.21.15.98:0/2839739141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7141f6b00 0x7fd714156b80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.076+0000 7fd7195c2640 1 --2- 172.21.15.98:0/2839739141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd700004d40 0x7fd714156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.076+0000 7fd7195c2640 1 -- 172.21.15.98:0/2839739141 >> 172.21.15.98:0/2839739141 conn(0x7fd7140d1d70 msgr2=0x7fd7140cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:23.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.076+0000 7fd7195c2640 1 -- 172.21.15.98:0/2839739141 shutdown_connections 2024-09-10T20:59:23.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.076+0000 7fd7195c2640 1 -- 172.21.15.98:0/2839739141 wait complete. 2024-09-10T20:59:23.092 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T20:59:23.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.192+0000 7f3f0cfad640 1 -- 172.21.15.98:0/580833395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f080d16f0 msgr2=0x7f3f0806ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.192+0000 7f3f0cfad640 1 --2- 172.21.15.98:0/580833395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f080d16f0 0x7f3f0806ce30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3f08075fb0 tx=0x7f3ef002a530 comp rx=0 tx=0).stop 2024-09-10T20:59:23.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.192+0000 7f3f0cfad640 1 -- 172.21.15.98:0/580833395 shutdown_connections 2024-09-10T20:59:23.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.192+0000 7f3f0cfad640 1 --2- 172.21.15.98:0/580833395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f080d16f0 0x7f3f0806ce30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.192+0000 7f3f0cfad640 1 --2- 172.21.15.98:0/580833395 >> v2:172.21.15.190:3300/0 conn(0x7f3f0806ca50 0x7f3f08070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.192+0000 7f3f0cfad640 1 -- 172.21.15.98:0/580833395 >> 172.21.15.98:0/580833395 conn(0x7f3f080d0c00 msgr2=0x7f3f080d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:23.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.192+0000 7f3f0cfad640 1 -- 172.21.15.98:0/580833395 shutdown_connections 2024-09-10T20:59:23.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.192+0000 7f3f0cfad640 1 -- 172.21.15.98:0/580833395 wait complete. 2024-09-10T20:59:23.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.193+0000 7f3f0cfad640 1 Processor -- start 2024-09-10T20:59:23.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.193+0000 7f3f0cfad640 1 -- start start 2024-09-10T20:59:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.194+0000 7f3f0cfad640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f0806ca50 0x7f3f081518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.194+0000 7f3f0cfad640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f08073670 0x7f3f0814eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.194+0000 7f3f0cfad640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f080d16f0 0x7f3f0814da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.194+0000 7f3f0cfad640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3f08145bf0 con 0x7f3f080d16f0 2024-09-10T20:59:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.194+0000 7f3f0cfad640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3f080cfc10 con 0x7f3f0806ca50 2024-09-10T20:59:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.194+0000 7f3f0cfad640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3f080cfd90 con 0x7f3f08073670 2024-09-10T20:59:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.194+0000 7f3f06d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f080d16f0 0x7f3f0814da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.194+0000 7f3f06d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f080d16f0 0x7f3f0814da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39360/0 (socket says 172.21.15.98:39360) 2024-09-10T20:59:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.194+0000 7f3f06d9d640 1 -- 172.21.15.98:0/3897692883 learned_addr learned my addr 172.21.15.98:0/3897692883 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.195+0000 7f3f0659c640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f0806ca50 0x7f3f081518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.195+0000 7f3efdd9b640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f08073670 0x7f3f0814eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.195+0000 7f3f06d9d640 1 -- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f08073670 msgr2=0x7f3f0814eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.195+0000 7f3f06d9d640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f08073670 0x7f3f0814eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.195+0000 7f3f06d9d640 1 -- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f0806ca50 msgr2=0x7f3f081518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.195+0000 7f3f06d9d640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f0806ca50 0x7f3f081518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.195+0000 7f3f06d9d640 1 -- 172.21.15.98:0/3897692883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0814f400 con 0x7f3f080d16f0 2024-09-10T20:59:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.195+0000 7f3efdd9b640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f08073670 0x7f3f0814eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.195+0000 7f3f06d9d640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f080d16f0 0x7f3f0814da60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c770 tx=0x7f3f0000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.195+0000 7f3eff7fe640 1 -- 172.21.15.98:0/3897692883 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00017070 con 0x7f3f080d16f0 2024-09-10T20:59:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.195+0000 7f3eff7fe640 1 -- 172.21.15.98:0/3897692883 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f00005070 con 0x7f3f080d16f0 2024-09-10T20:59:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.195+0000 7f3f0cfad640 1 -- 172.21.15.98:0/3897692883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0814c9c0 con 0x7f3f080d16f0 2024-09-10T20:59:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.195+0000 7f3eff7fe640 1 -- 172.21.15.98:0/3897692883 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f000054c0 con 0x7f3f080d16f0 2024-09-10T20:59:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.195+0000 7f3f0cfad640 1 -- 172.21.15.98:0/3897692883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f0814cf80 con 0x7f3f080d16f0 2024-09-10T20:59:23.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.196+0000 7f3f0cfad640 1 -- 172.21.15.98:0/3897692883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f08074370 con 0x7f3f080d16f0 2024-09-10T20:59:23.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.197+0000 7f3eff7fe640 1 -- 172.21.15.98:0/3897692883 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3f00005660 con 0x7f3f080d16f0 2024-09-10T20:59:23.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.197+0000 7f3eff7fe640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3ee8077f70 0x7f3ee807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:23.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.197+0000 7f3eff7fe640 1 -- 172.21.15.98:0/3897692883 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3f00099a20 con 0x7f3f080d16f0 2024-09-10T20:59:23.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.197+0000 7f3f0659c640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3ee8077f70 0x7f3ee807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:23.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.198+0000 7f3f0659c640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3ee8077f70 0x7f3ee807a430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8005d80 tx=0x7f3ef8005c50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:23.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.201+0000 7f3eff7fe640 1 -- 172.21.15.98:0/3897692883 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3f0814f400 con 0x7f3f080d16f0 2024-09-10T20:59:23.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.379+0000 7f3f0cfad640 1 -- 172.21.15.98:0/3897692883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3f080ce7b0 con 0x7f3f080d16f0 2024-09-10T20:59:23.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.380+0000 7f3eff7fe640 1 -- 172.21.15.98:0/3897692883 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f3f000621e0 con 0x7f3f080d16f0 2024-09-10T20:59:23.384 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-09-10T20:59:23.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.382+0000 7f3f0cfad640 1 -- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3ee8077f70 msgr2=0x7f3ee807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.382+0000 7f3f0cfad640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3ee8077f70 0x7f3ee807a430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8005d80 tx=0x7f3ef8005c50 comp rx=0 tx=0).stop 2024-09-10T20:59:23.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.382+0000 7f3f0cfad640 1 -- 172.21.15.98:0/3897692883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f080d16f0 msgr2=0x7f3f0814da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.382+0000 7f3f0cfad640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f080d16f0 0x7f3f0814da60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000c770 tx=0x7f3f0000cc30 comp rx=0 tx=0).stop 2024-09-10T20:59:23.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.382+0000 7f3f0cfad640 1 -- 172.21.15.98:0/3897692883 shutdown_connections 2024-09-10T20:59:23.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.382+0000 7f3f0cfad640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3ee8077f70 0x7f3ee807a430 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.382+0000 7f3f0cfad640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f080d16f0 0x7f3f0814da60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.382+0000 7f3f0cfad640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f08073670 0x7f3f0814eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.382+0000 7f3f0cfad640 1 --2- 172.21.15.98:0/3897692883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f0806ca50 0x7f3f081518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.382+0000 7f3f0cfad640 1 -- 172.21.15.98:0/3897692883 >> 172.21.15.98:0/3897692883 conn(0x7f3f080d0c00 msgr2=0x7f3f0815ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:23.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.382+0000 7f3f0cfad640 1 -- 172.21.15.98:0/3897692883 shutdown_connections 2024-09-10T20:59:23.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.382+0000 7f3f0cfad640 1 -- 172.21.15.98:0/3897692883 wait complete. 2024-09-10T20:59:23.397 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T20:59:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.496+0000 7f622c82a640 1 -- 172.21.15.98:0/1172458877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f622806ca50 msgr2=0x7f6228070910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.496+0000 7f622c82a640 1 --2- 172.21.15.98:0/1172458877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f622806ca50 0x7f6228070910 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f62280b3c00 tx=0x7f621802a8b0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.496+0000 7f622c82a640 1 -- 172.21.15.98:0/1172458877 shutdown_connections 2024-09-10T20:59:23.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.496+0000 7f622c82a640 1 --2- 172.21.15.98:0/1172458877 >> v2:172.21.15.190:3300/0 conn(0x7f62280d16f0 0x7f622806ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.496+0000 7f622c82a640 1 --2- 172.21.15.98:0/1172458877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f622806ca50 0x7f6228070910 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.496+0000 7f622c82a640 1 -- 172.21.15.98:0/1172458877 >> 172.21.15.98:0/1172458877 conn(0x7f62280d0c00 msgr2=0x7f62280d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:23.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.496+0000 7f622c82a640 1 -- 172.21.15.98:0/1172458877 shutdown_connections 2024-09-10T20:59:23.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.496+0000 7f622c82a640 1 -- 172.21.15.98:0/1172458877 wait complete. 2024-09-10T20:59:23.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.498+0000 7f622c82a640 1 Processor -- start 2024-09-10T20:59:23.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.498+0000 7f622c82a640 1 -- start start 2024-09-10T20:59:23.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.498+0000 7f622c82a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622806ca50 0x7f62280b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:23.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.498+0000 7f622c82a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6228073670 0x7f62281e64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:23.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.498+0000 7f622c82a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62280d16f0 0x7f62281e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:23.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.498+0000 7f622c82a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6228145bf0 con 0x7f6228073670 2024-09-10T20:59:23.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.498+0000 7f622c82a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f62280cfc10 con 0x7f622806ca50 2024-09-10T20:59:23.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.498+0000 7f622c82a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f62280cfd90 con 0x7f62280d16f0 2024-09-10T20:59:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.498+0000 7f6225d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6228073670 0x7f62281e64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.498+0000 7f6225d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6228073670 0x7f62281e64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39376/0 (socket says 172.21.15.98:39376) 2024-09-10T20:59:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.498+0000 7f6225d9b640 1 -- 172.21.15.98:0/2109644772 learned_addr learned my addr 172.21.15.98:0/2109644772 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.499+0000 7f622659c640 1 --2- 172.21.15.98:0/2109644772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622806ca50 0x7f62280b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.499+0000 7f6226d9d640 1 --2- 172.21.15.98:0/2109644772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62280d16f0 0x7f62281e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.499+0000 7f6225d9b640 1 -- 172.21.15.98:0/2109644772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62280d16f0 msgr2=0x7f62281e6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.499+0000 7f6225d9b640 1 --2- 172.21.15.98:0/2109644772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62280d16f0 0x7f62281e6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.499+0000 7f6225d9b640 1 -- 172.21.15.98:0/2109644772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622806ca50 msgr2=0x7f62280b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.499+0000 7f6225d9b640 1 --2- 172.21.15.98:0/2109644772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622806ca50 0x7f62280b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.499+0000 7f6225d9b640 1 -- 172.21.15.98:0/2109644772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6228159640 con 0x7f6228073670 2024-09-10T20:59:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.499+0000 7f6225d9b640 1 --2- 172.21.15.98:0/2109644772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6228073670 0x7f62281e64f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f621000d740 tx=0x7f621000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.499+0000 7f621effd640 1 -- 172.21.15.98:0/2109644772 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6210017070 con 0x7f6228073670 2024-09-10T20:59:23.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.499+0000 7f621effd640 1 -- 172.21.15.98:0/2109644772 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6210009d20 con 0x7f6228073670 2024-09-10T20:59:23.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.499+0000 7f621effd640 1 -- 172.21.15.98:0/2109644772 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62100106c0 con 0x7f6228073670 2024-09-10T20:59:23.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.499+0000 7f622c82a640 1 -- 172.21.15.98:0/2109644772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6228159920 con 0x7f6228073670 2024-09-10T20:59:23.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.499+0000 7f622c82a640 1 -- 172.21.15.98:0/2109644772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f622815a400 con 0x7f6228073670 2024-09-10T20:59:23.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.500+0000 7f622c82a640 1 -- 172.21.15.98:0/2109644772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6228074370 con 0x7f6228073670 2024-09-10T20:59:23.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.501+0000 7f621effd640 1 -- 172.21.15.98:0/2109644772 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6210010860 con 0x7f6228073670 2024-09-10T20:59:23.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.501+0000 7f621effd640 1 --2- 172.21.15.98:0/2109644772 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6208077f70 0x7f620807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:23.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.501+0000 7f621effd640 1 -- 172.21.15.98:0/2109644772 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f621009a440 con 0x7f6228073670 2024-09-10T20:59:23.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.501+0000 7f622659c640 1 --2- 172.21.15.98:0/2109644772 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6208077f70 0x7f620807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:23.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.502+0000 7f622659c640 1 --2- 172.21.15.98:0/2109644772 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6208077f70 0x7f620807a430 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6218034000 tx=0x7f6218004ab0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:23.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.505+0000 7f621effd640 1 -- 172.21.15.98:0/2109644772 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f62100625b0 con 0x7f6228073670 2024-09-10T20:59:23.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.681+0000 7f622c82a640 1 -- 172.21.15.98:0/2109644772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f62280ce7b0 con 0x7f6228073670 2024-09-10T20:59:23.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.681+0000 7f621effd640 1 -- 172.21.15.98:0/2109644772 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f6210066fe0 con 0x7f6228073670 2024-09-10T20:59:23.686 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-09-10T20:59:23.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.683+0000 7f622c82a640 1 -- 172.21.15.98:0/2109644772 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6208077f70 msgr2=0x7f620807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.683+0000 7f622c82a640 1 --2- 172.21.15.98:0/2109644772 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6208077f70 0x7f620807a430 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6218034000 tx=0x7f6218004ab0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.683+0000 7f622c82a640 1 -- 172.21.15.98:0/2109644772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6228073670 msgr2=0x7f62281e64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.683+0000 7f622c82a640 1 --2- 172.21.15.98:0/2109644772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6228073670 0x7f62281e64f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f621000d740 tx=0x7f621000dc00 comp rx=0 tx=0).stop 2024-09-10T20:59:23.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.683+0000 7f622c82a640 1 -- 172.21.15.98:0/2109644772 shutdown_connections 2024-09-10T20:59:23.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.683+0000 7f622c82a640 1 --2- 172.21.15.98:0/2109644772 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6208077f70 0x7f620807a430 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.683+0000 7f622c82a640 1 --2- 172.21.15.98:0/2109644772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62280d16f0 0x7f62281e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.683+0000 7f622c82a640 1 --2- 172.21.15.98:0/2109644772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6228073670 0x7f62281e64f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.683+0000 7f622c82a640 1 --2- 172.21.15.98:0/2109644772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622806ca50 0x7f62280b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.683+0000 7f622c82a640 1 -- 172.21.15.98:0/2109644772 >> 172.21.15.98:0/2109644772 conn(0x7f62280d0c00 msgr2=0x7f622813cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:23.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.683+0000 7f622c82a640 1 -- 172.21.15.98:0/2109644772 shutdown_connections 2024-09-10T20:59:23.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.683+0000 7f622c82a640 1 -- 172.21.15.98:0/2109644772 wait complete. 2024-09-10T20:59:23.699 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create' 2024-09-10T20:59:23.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.796+0000 7faa9b39b640 1 -- 172.21.15.98:0/2569127328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9406d160 msgr2=0x7faa940b1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.796+0000 7faa9b39b640 1 --2- 172.21.15.98:0/2569127328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9406d160 0x7faa940b1210 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faa84005fd0 tx=0x7faa8402a560 comp rx=0 tx=0).stop 2024-09-10T20:59:23.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.796+0000 7faa9b39b640 1 -- 172.21.15.98:0/2569127328 shutdown_connections 2024-09-10T20:59:23.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.796+0000 7faa9b39b640 1 --2- 172.21.15.98:0/2569127328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa94074c70 0x7faa940cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.796+0000 7faa9b39b640 1 --2- 172.21.15.98:0/2569127328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9406d160 0x7faa940b1210 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.796+0000 7faa9b39b640 1 -- 172.21.15.98:0/2569127328 >> 172.21.15.98:0/2569127328 conn(0x7faa94074150 msgr2=0x7faa94074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:23.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.796+0000 7faa9b39b640 1 -- 172.21.15.98:0/2569127328 shutdown_connections 2024-09-10T20:59:23.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.796+0000 7faa9b39b640 1 -- 172.21.15.98:0/2569127328 wait complete. 2024-09-10T20:59:23.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.797+0000 7faa9b39b640 1 Processor -- start 2024-09-10T20:59:23.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.797+0000 7faa9b39b640 1 -- start start 2024-09-10T20:59:23.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa9b39b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9406d160 0x7faa9415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:23.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa9b39b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa94074c70 0x7faa9415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:23.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa9b39b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa9415e0f0 0x7faa9415bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:23.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa9b39b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faa940697b0 con 0x7faa9406d160 2024-09-10T20:59:23.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa9b39b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faa94069630 con 0x7faa94074c70 2024-09-10T20:59:23.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa9b39b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7faa940ce430 con 0x7faa9415e0f0 2024-09-10T20:59:23.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa99137640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9406d160 0x7faa9415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:23.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa99137640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9406d160 0x7faa9415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39396/0 (socket says 172.21.15.98:39396) 2024-09-10T20:59:23.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa99137640 1 -- 172.21.15.98:0/4128095293 learned_addr learned my addr 172.21.15.98:0/4128095293 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:23.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa99137640 1 -- 172.21.15.98:0/4128095293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa9415e0f0 msgr2=0x7faa9415bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa99137640 1 --2- 172.21.15.98:0/4128095293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa9415e0f0 0x7faa9415bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa99137640 1 -- 172.21.15.98:0/4128095293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa94074c70 msgr2=0x7faa9415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:23.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa99137640 1 --2- 172.21.15.98:0/4128095293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa94074c70 0x7faa9415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:23.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa99137640 1 -- 172.21.15.98:0/4128095293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa941f46c0 con 0x7faa9406d160 2024-09-10T20:59:23.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa99137640 1 --2- 172.21.15.98:0/4128095293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9406d160 0x7faa9415b1f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faa9000ba50 tx=0x7faa9000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:23.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.798+0000 7faa89ffb640 1 -- 172.21.15.98:0/4128095293 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa90019070 con 0x7faa9406d160 2024-09-10T20:59:23.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.799+0000 7faa89ffb640 1 -- 172.21.15.98:0/4128095293 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa9000cd10 con 0x7faa9406d160 2024-09-10T20:59:23.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.799+0000 7faa9b39b640 1 -- 172.21.15.98:0/4128095293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa941f4960 con 0x7faa9406d160 2024-09-10T20:59:23.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.799+0000 7faa89ffb640 1 -- 172.21.15.98:0/4128095293 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa900146a0 con 0x7faa9406d160 2024-09-10T20:59:23.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.799+0000 7faa9b39b640 1 -- 172.21.15.98:0/4128095293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa941f4f20 con 0x7faa9406d160 2024-09-10T20:59:23.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.800+0000 7faa9b39b640 1 -- 172.21.15.98:0/4128095293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa940ce970 con 0x7faa9406d160 2024-09-10T20:59:23.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.800+0000 7faa89ffb640 1 -- 172.21.15.98:0/4128095293 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7faa90014840 con 0x7faa9406d160 2024-09-10T20:59:23.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.801+0000 7faa89ffb640 1 --2- 172.21.15.98:0/4128095293 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7faa78077f70 0x7faa7807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:23.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.801+0000 7faa89ffb640 1 -- 172.21.15.98:0/4128095293 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faa90099f90 con 0x7faa9406d160 2024-09-10T20:59:23.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.801+0000 7faa98936640 1 --2- 172.21.15.98:0/4128095293 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7faa78077f70 0x7faa7807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:23.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.802+0000 7faa98936640 1 --2- 172.21.15.98:0/4128095293 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7faa78077f70 0x7faa7807a430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa84033000 tx=0x7faa840046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:23.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:23.805+0000 7faa89ffb640 1 -- 172.21.15.98:0/4128095293 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7faa90062190 con 0x7faa9406d160 2024-09-10T20:59:24.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.047+0000 7faa9b39b640 1 -- 172.21.15.98:0/4128095293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]} v 0) -- 0x7faa9415d7a0 con 0x7faa9406d160 2024-09-10T20:59:24.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.362+0000 7faa89ffb640 1 -- 172.21.15.98:0/4128095293 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]}]=0 v58) ==== 141+0+0 (secure 0 0 0) 0x7faa90066bc0 con 0x7faa9406d160 2024-09-10T20:59:24.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.365+0000 7faa9b39b640 1 -- 172.21.15.98:0/4128095293 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7faa78077f70 msgr2=0x7faa7807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:24.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.365+0000 7faa9b39b640 1 --2- 172.21.15.98:0/4128095293 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7faa78077f70 0x7faa7807a430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa84033000 tx=0x7faa840046c0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.365+0000 7faa9b39b640 1 -- 172.21.15.98:0/4128095293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9406d160 msgr2=0x7faa9415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:24.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.365+0000 7faa9b39b640 1 --2- 172.21.15.98:0/4128095293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9406d160 0x7faa9415b1f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faa9000ba50 tx=0x7faa9000bf10 comp rx=0 tx=0).stop 2024-09-10T20:59:24.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.365+0000 7faa9b39b640 1 -- 172.21.15.98:0/4128095293 shutdown_connections 2024-09-10T20:59:24.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.365+0000 7faa9b39b640 1 --2- 172.21.15.98:0/4128095293 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7faa78077f70 0x7faa7807a430 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.365+0000 7faa9b39b640 1 --2- 172.21.15.98:0/4128095293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa9415e0f0 0x7faa9415bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.365+0000 7faa9b39b640 1 --2- 172.21.15.98:0/4128095293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa94074c70 0x7faa9415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.365+0000 7faa9b39b640 1 --2- 172.21.15.98:0/4128095293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9406d160 0x7faa9415b1f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.365+0000 7faa9b39b640 1 -- 172.21.15.98:0/4128095293 >> 172.21.15.98:0/4128095293 conn(0x7faa94074150 msgr2=0x7faa94074530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:24.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.365+0000 7faa9b39b640 1 -- 172.21.15.98:0/4128095293 shutdown_connections 2024-09-10T20:59:24.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.365+0000 7faa9b39b640 1 -- 172.21.15.98:0/4128095293 wait complete. 2024-09-10T20:59:24.380 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T20:59:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.480+0000 7fb835575640 1 -- 172.21.15.98:0/712831738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8300d2830 msgr2=0x7fb8300cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.480+0000 7fb835575640 1 --2- 172.21.15.98:0/712831738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8300d2830 0x7fb8300cf8e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb828005fd0 tx=0x7fb82802a8b0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.480+0000 7fb835575640 1 -- 172.21.15.98:0/712831738 shutdown_connections 2024-09-10T20:59:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.480+0000 7fb835575640 1 --2- 172.21.15.98:0/712831738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8300d2830 0x7fb8300cf8e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.480+0000 7fb835575640 1 --2- 172.21.15.98:0/712831738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8300d0a00 0x7fb830070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.480+0000 7fb835575640 1 -- 172.21.15.98:0/712831738 >> 172.21.15.98:0/712831738 conn(0x7fb8300d1d10 msgr2=0x7fb8300d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.480+0000 7fb835575640 1 -- 172.21.15.98:0/712831738 shutdown_connections 2024-09-10T20:59:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.480+0000 7fb835575640 1 -- 172.21.15.98:0/712831738 wait complete. 2024-09-10T20:59:24.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.482+0000 7fb835575640 1 Processor -- start 2024-09-10T20:59:24.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.482+0000 7fb835575640 1 -- start start 2024-09-10T20:59:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.482+0000 7fb835575640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8300d0a00 0x7fb8300c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.482+0000 7fb835575640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8300d2830 0x7fb8300c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.482+0000 7fb835575640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb83015be00 0x7fb8301efd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.482+0000 7fb835575640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb8300cd1d0 con 0x7fb8300d0a00 2024-09-10T20:59:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.482+0000 7fb835575640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb8300cd050 con 0x7fb83015be00 2024-09-10T20:59:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.482+0000 7fb835575640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb8300cd350 con 0x7fb8300d2830 2024-09-10T20:59:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb82effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8300d0a00 0x7fb8300c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb82effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8300d0a00 0x7fb8300c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39412/0 (socket says 172.21.15.98:39412) 2024-09-10T20:59:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb82effd640 1 -- 172.21.15.98:0/3234716586 learned_addr learned my addr 172.21.15.98:0/3234716586 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb82f7fe640 1 --2- 172.21.15.98:0/3234716586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb83015be00 0x7fb8301efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb82e7fc640 1 --2- 172.21.15.98:0/3234716586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8300d2830 0x7fb8300c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb82effd640 1 -- 172.21.15.98:0/3234716586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8300d2830 msgr2=0x7fb8300c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb82effd640 1 --2- 172.21.15.98:0/3234716586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8300d2830 0x7fb8300c6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb82effd640 1 -- 172.21.15.98:0/3234716586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb83015be00 msgr2=0x7fb8301efd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb82effd640 1 --2- 172.21.15.98:0/3234716586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb83015be00 0x7fb8301efd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb82effd640 1 -- 172.21.15.98:0/3234716586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8300c6790 con 0x7fb8300d0a00 2024-09-10T20:59:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb82effd640 1 --2- 172.21.15.98:0/3234716586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8300d0a00 0x7fb8300c8050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb82000c370 tx=0x7fb82000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb8177fe640 1 -- 172.21.15.98:0/3234716586 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820006890 con 0x7fb8300d0a00 2024-09-10T20:59:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb8177fe640 1 -- 172.21.15.98:0/3234716586 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb820006b20 con 0x7fb8300d0a00 2024-09-10T20:59:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb835575640 1 -- 172.21.15.98:0/3234716586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8301f6de0 con 0x7fb8300d0a00 2024-09-10T20:59:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb8177fe640 1 -- 172.21.15.98:0/3234716586 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820007020 con 0x7fb8300d0a00 2024-09-10T20:59:24.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.483+0000 7fb835575640 1 -- 172.21.15.98:0/3234716586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8300c7290 con 0x7fb8300d0a00 2024-09-10T20:59:24.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.484+0000 7fb835575640 1 -- 172.21.15.98:0/3234716586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb830070760 con 0x7fb8300d0a00 2024-09-10T20:59:24.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.485+0000 7fb8177fe640 1 -- 172.21.15.98:0/3234716586 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb8200071c0 con 0x7fb8300d0a00 2024-09-10T20:59:24.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.485+0000 7fb8177fe640 1 --2- 172.21.15.98:0/3234716586 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fb81c077f70 0x7fb81c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:24.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.486+0000 7fb8177fe640 1 -- 172.21.15.98:0/3234716586 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb820099940 con 0x7fb8300d0a00 2024-09-10T20:59:24.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.486+0000 7fb82e7fc640 1 --2- 172.21.15.98:0/3234716586 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fb81c077f70 0x7fb81c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:24.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.486+0000 7fb82e7fc640 1 --2- 172.21.15.98:0/3234716586 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fb81c077f70 0x7fb81c07a430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb8300c8590 tx=0x7fb824005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:24.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.490+0000 7fb8177fe640 1 -- 172.21.15.98:0/3234716586 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb8300c6790 con 0x7fb8300d0a00 2024-09-10T20:59:24.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.673+0000 7fb835575640 1 -- 172.21.15.98:0/3234716586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fb8300cdf30 con 0x7fb8300d0a00 2024-09-10T20:59:24.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.674+0000 7fb8177fe640 1 -- 172.21.15.98:0/3234716586 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v27) ==== 59+0+1998 (secure 0 0 0) 0x7fb820061ac0 con 0x7fb8300d0a00 2024-09-10T20:59:24.678 INFO:teuthology.orchestra.run.smithi098.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T20:59:24.678 INFO:teuthology.orchestra.run.smithi098.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T20:59:24.678 INFO:teuthology.orchestra.run.smithi098.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T20:59:24.678 INFO:teuthology.orchestra.run.smithi098.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T20:59:24.678 INFO:teuthology.orchestra.run.smithi098.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 6410.658734 2024-09-10T20:59:24.679 INFO:teuthology.orchestra.run.smithi098.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 5192.223531 2024-09-10T20:59:24.679 INFO:teuthology.orchestra.run.smithi098.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 5123.311936 2024-09-10T20:59:24.679 INFO:teuthology.orchestra.run.smithi098.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 6515.175332 2024-09-10T20:59:24.679 INFO:teuthology.orchestra.run.smithi098.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6063.662514 2024-09-10T20:59:24.679 INFO:teuthology.orchestra.run.smithi098.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 9131.186061 2024-09-10T20:59:24.679 INFO:teuthology.orchestra.run.smithi098.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5603.346780 2024-09-10T20:59:24.679 INFO:teuthology.orchestra.run.smithi098.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5704.429210 2024-09-10T20:59:24.679 INFO:teuthology.orchestra.run.smithi098.stdout:mds advanced debug_mds 20/20 2024-09-10T20:59:24.679 INFO:teuthology.orchestra.run.smithi098.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T20:59:24.679 INFO:teuthology.orchestra.run.smithi098.stdout:mds advanced debug_ms 2/2 2024-09-10T20:59:24.679 INFO:teuthology.orchestra.run.smithi098.stdout:mds dev mds_debug_frag true 2024-09-10T20:59:24.679 INFO:teuthology.orchestra.run.smithi098.stdout:mds dev mds_debug_scatterstat true 2024-09-10T20:59:24.679 INFO:teuthology.orchestra.run.smithi098.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T20:59:24.680 INFO:teuthology.orchestra.run.smithi098.stdout:mds dev mds_verify_scatter true 2024-09-10T20:59:24.680 INFO:teuthology.orchestra.run.smithi098.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T20:59:24.680 INFO:teuthology.orchestra.run.smithi098.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T20:59:24.680 INFO:teuthology.orchestra.run.smithi098.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T20:59:24.680 INFO:teuthology.orchestra.run.smithi098.stdout:client advanced client_mount_timeout 600 2024-09-10T20:59:24.680 INFO:teuthology.orchestra.run.smithi098.stdout:client advanced debug_client 20/20 2024-09-10T20:59:24.680 INFO:teuthology.orchestra.run.smithi098.stdout:client advanced debug_ms 1/1 2024-09-10T20:59:24.680 INFO:teuthology.orchestra.run.smithi098.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T20:59:24.680 INFO:teuthology.orchestra.run.smithi098.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T20:59:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.676+0000 7fb835575640 1 -- 172.21.15.98:0/3234716586 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fb81c077f70 msgr2=0x7fb81c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.676+0000 7fb835575640 1 --2- 172.21.15.98:0/3234716586 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fb81c077f70 0x7fb81c07a430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb8300c8590 tx=0x7fb824005f90 comp rx=0 tx=0).stop 2024-09-10T20:59:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.676+0000 7fb835575640 1 -- 172.21.15.98:0/3234716586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8300d0a00 msgr2=0x7fb8300c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.676+0000 7fb835575640 1 --2- 172.21.15.98:0/3234716586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8300d0a00 0x7fb8300c8050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb82000c370 tx=0x7fb82000c830 comp rx=0 tx=0).stop 2024-09-10T20:59:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.676+0000 7fb835575640 1 -- 172.21.15.98:0/3234716586 shutdown_connections 2024-09-10T20:59:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.676+0000 7fb835575640 1 --2- 172.21.15.98:0/3234716586 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fb81c077f70 0x7fb81c07a430 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.676+0000 7fb835575640 1 --2- 172.21.15.98:0/3234716586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb83015be00 0x7fb8301efd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.676+0000 7fb835575640 1 --2- 172.21.15.98:0/3234716586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8300d2830 0x7fb8300c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.676+0000 7fb835575640 1 --2- 172.21.15.98:0/3234716586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8300d0a00 0x7fb8300c8050 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.676+0000 7fb835575640 1 -- 172.21.15.98:0/3234716586 >> 172.21.15.98:0/3234716586 conn(0x7fb8300d1d10 msgr2=0x7fb8300cd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.676+0000 7fb835575640 1 -- 172.21.15.98:0/3234716586 shutdown_connections 2024-09-10T20:59:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.676+0000 7fb835575640 1 -- 172.21.15.98:0/3234716586 wait complete. 2024-09-10T20:59:24.691 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T20:59:24.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.793+0000 7f010a09b640 1 -- 172.21.15.98:0/3139559343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0104144780 msgr2=0x7f01040b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:24.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.793+0000 7f010a09b640 1 --2- 172.21.15.98:0/3139559343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0104144780 0x7f01040b3da0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f01040749f0 tx=0x7f00f402a4d0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.793+0000 7f010a09b640 1 -- 172.21.15.98:0/3139559343 shutdown_connections 2024-09-10T20:59:24.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.793+0000 7f010a09b640 1 --2- 172.21.15.98:0/3139559343 >> v2:172.21.15.190:3300/0 conn(0x7f01040ceb00 0x7f01040cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.793+0000 7f010a09b640 1 --2- 172.21.15.98:0/3139559343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0104072680 0x7f0104072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.793+0000 7f010a09b640 1 --2- 172.21.15.98:0/3139559343 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0104144780 0x7f01040b3da0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.793+0000 7f010a09b640 1 -- 172.21.15.98:0/3139559343 >> 172.21.15.98:0/3139559343 conn(0x7f01040b1760 msgr2=0x7f01040b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:24.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.793+0000 7f010a09b640 1 -- 172.21.15.98:0/3139559343 shutdown_connections 2024-09-10T20:59:24.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.793+0000 7f010a09b640 1 -- 172.21.15.98:0/3139559343 wait complete. 2024-09-10T20:59:24.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.795+0000 7f010a09b640 1 Processor -- start 2024-09-10T20:59:24.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.795+0000 7f010a09b640 1 -- start start 2024-09-10T20:59:24.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.796+0000 7f010a09b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0104072680 0x7f01041f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:24.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.796+0000 7f010a09b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01040ceb00 0x7f01041f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:24.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.796+0000 7f010a09b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0104144780 0x7f01041f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:24.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.796+0000 7f010a09b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f010415c8e0 con 0x7f01040ceb00 2024-09-10T20:59:24.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.796+0000 7f010a09b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f010415c760 con 0x7f0104072680 2024-09-10T20:59:24.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.796+0000 7f010a09b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f010415ca60 con 0x7f0104144780 2024-09-10T20:59:24.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.796+0000 7f0102ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01040ceb00 0x7f01041f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:24.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.796+0000 7f0102ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01040ceb00 0x7f01041f53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39426/0 (socket says 172.21.15.98:39426) 2024-09-10T20:59:24.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.796+0000 7f0102ffd640 1 -- 172.21.15.98:0/1301983471 learned_addr learned my addr 172.21.15.98:0/1301983471 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:24.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.796+0000 7f0103fff640 1 --2- 172.21.15.98:0/1301983471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0104144780 0x7f01041f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:24.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.797+0000 7f01037fe640 1 --2- 172.21.15.98:0/1301983471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0104072680 0x7f01041f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:24.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.797+0000 7f0102ffd640 1 -- 172.21.15.98:0/1301983471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0104144780 msgr2=0x7f01041f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:24.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.797+0000 7f0102ffd640 1 --2- 172.21.15.98:0/1301983471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0104144780 0x7f01041f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.797+0000 7f0102ffd640 1 -- 172.21.15.98:0/1301983471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0104072680 msgr2=0x7f01041f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:24.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.797+0000 7f0102ffd640 1 --2- 172.21.15.98:0/1301983471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0104072680 0x7f01041f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.797+0000 7f0102ffd640 1 -- 172.21.15.98:0/1301983471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0104140e00 con 0x7f01040ceb00 2024-09-10T20:59:24.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.797+0000 7f0102ffd640 1 --2- 172.21.15.98:0/1301983471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01040ceb00 0x7f01041f53f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00b4d0 tx=0x7f00ec00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:24.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.797+0000 7f00fbfff640 1 -- 172.21.15.98:0/1301983471 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec014070 con 0x7f01040ceb00 2024-09-10T20:59:24.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.797+0000 7f00fbfff640 1 -- 172.21.15.98:0/1301983471 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00ec010960 con 0x7f01040ceb00 2024-09-10T20:59:24.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.797+0000 7f00fbfff640 1 -- 172.21.15.98:0/1301983471 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec010db0 con 0x7f01040ceb00 2024-09-10T20:59:24.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.797+0000 7f010a09b640 1 -- 172.21.15.98:0/1301983471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01041f2c40 con 0x7f01040ceb00 2024-09-10T20:59:24.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.797+0000 7f010a09b640 1 -- 172.21.15.98:0/1301983471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01041f1d80 con 0x7f01040ceb00 2024-09-10T20:59:24.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.798+0000 7f010a09b640 1 -- 172.21.15.98:0/1301983471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010415cf80 con 0x7f01040ceb00 2024-09-10T20:59:24.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.799+0000 7f00fbfff640 1 -- 172.21.15.98:0/1301983471 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f00ec00f3c0 con 0x7f01040ceb00 2024-09-10T20:59:24.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.799+0000 7f00fbfff640 1 --2- 172.21.15.98:0/1301983471 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f00e4077f70 0x7f00e407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:24.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.799+0000 7f00fbfff640 1 -- 172.21.15.98:0/1301983471 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f00ec09a0e0 con 0x7f01040ceb00 2024-09-10T20:59:24.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.799+0000 7f01037fe640 1 --2- 172.21.15.98:0/1301983471 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f00e4077f70 0x7f00e407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:24.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.800+0000 7f01037fe640 1 --2- 172.21.15.98:0/1301983471 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f00e4077f70 0x7f00e407a430 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f00f4033000 tx=0x7f00f4004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:24.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.803+0000 7f00fbfff640 1 -- 172.21.15.98:0/1301983471 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f00ec062250 con 0x7f01040ceb00 2024-09-10T20:59:24.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.980+0000 7f010a09b640 1 -- 172.21.15.98:0/1301983471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f01040b4d40 con 0x7f01040ceb00 2024-09-10T20:59:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.983+0000 7f00fbfff640 1 -- 172.21.15.98:0/1301983471 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f00ec066c80 con 0x7f01040ceb00 2024-09-10T20:59:24.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.986+0000 7f010a09b640 1 -- 172.21.15.98:0/1301983471 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f00e4077f70 msgr2=0x7f00e407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:24.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.986+0000 7f010a09b640 1 --2- 172.21.15.98:0/1301983471 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f00e4077f70 0x7f00e407a430 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f00f4033000 tx=0x7f00f4004730 comp rx=0 tx=0).stop 2024-09-10T20:59:24.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.986+0000 7f010a09b640 1 -- 172.21.15.98:0/1301983471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01040ceb00 msgr2=0x7f01041f53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:24.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.986+0000 7f010a09b640 1 --2- 172.21.15.98:0/1301983471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01040ceb00 0x7f01041f53f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00b4d0 tx=0x7f00ec00b990 comp rx=0 tx=0).stop 2024-09-10T20:59:24.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.986+0000 7f010a09b640 1 -- 172.21.15.98:0/1301983471 shutdown_connections 2024-09-10T20:59:24.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.986+0000 7f010a09b640 1 --2- 172.21.15.98:0/1301983471 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f00e4077f70 0x7f00e407a430 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.986+0000 7f010a09b640 1 --2- 172.21.15.98:0/1301983471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0104144780 0x7f01041f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.986+0000 7f010a09b640 1 --2- 172.21.15.98:0/1301983471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01040ceb00 0x7f01041f53f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.986+0000 7f010a09b640 1 --2- 172.21.15.98:0/1301983471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0104072680 0x7f01041f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.986+0000 7f010a09b640 1 -- 172.21.15.98:0/1301983471 >> 172.21.15.98:0/1301983471 conn(0x7f01040b1760 msgr2=0x7f01040cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:24.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.986+0000 7f010a09b640 1 -- 172.21.15.98:0/1301983471 shutdown_connections 2024-09-10T20:59:24.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:24.986+0000 7f010a09b640 1 -- 172.21.15.98:0/1301983471 wait complete. 2024-09-10T20:59:25.000 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T20:59:25.001 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T20:59:25.041 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T20:59:25.085 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T20:59:25.085 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:59:25.086 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:25.102 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:59:25.102 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:25.157 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:59:25.157 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:25.214 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T20:59:25.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.364+0000 7f69ef94b640 1 -- 172.21.15.98:0/2060927175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69d0004800 msgr2=0x7f69d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:25.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.364+0000 7f69ef94b640 1 --2- 172.21.15.98:0/2060927175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69d0004800 0x7f69d0004c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0016c0 tx=0x7f69d0005830 comp rx=0 tx=0).stop 2024-09-10T20:59:25.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.364+0000 7f69edee8640 1 -- 172.21.15.98:0/2060927175 reap_dead start 2024-09-10T20:59:25.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.364+0000 7f69ef94b640 1 -- 172.21.15.98:0/2060927175 shutdown_connections 2024-09-10T20:59:25.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.364+0000 7f69ef94b640 1 -- 172.21.15.98:0/2060927175 >> 172.21.15.98:0/2060927175 conn(0x7f69e80b1760 msgr2=0x7f69e80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:25.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.364+0000 7f69ef94b640 1 -- 172.21.15.98:0/2060927175 shutdown_connections 2024-09-10T20:59:25.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.364+0000 7f69ef94b640 1 -- 172.21.15.98:0/2060927175 wait complete. 2024-09-10T20:59:25.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.365+0000 7f69ef94b640 1 Processor -- start 2024-09-10T20:59:25.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.365+0000 7f69ef94b640 1 -- start start 2024-09-10T20:59:25.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69ef94b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d0004800 0x7f69e81394b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:25.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69ef94b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69e815c670 0x7f69e80b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:25.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69ef94b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69e814aab0 0x7f69e80b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:25.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69ef94b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f69e80b3d00 con 0x7f69e814aab0 2024-09-10T20:59:25.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69ef94b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f69e80b3b80 con 0x7f69e815c670 2024-09-10T20:59:25.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69ef94b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f69e80b3e80 con 0x7f69d0004800 2024-09-10T20:59:25.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69edee8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69e814aab0 0x7f69e80b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:25.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69edee8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69e814aab0 0x7f69e80b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39444/0 (socket says 172.21.15.98:39444) 2024-09-10T20:59:25.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69edee8640 1 -- 172.21.15.98:0/3907472472 learned_addr learned my addr 172.21.15.98:0/3907472472 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:25.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69ed6e7640 1 --2- 172.21.15.98:0/3907472472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d0004800 0x7f69e81394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:25.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69ecee6640 1 --2- 172.21.15.98:0/3907472472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69e815c670 0x7f69e80b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:25.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69edee8640 1 -- 172.21.15.98:0/3907472472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d0004800 msgr2=0x7f69e81394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:25.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69edee8640 1 --2- 172.21.15.98:0/3907472472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d0004800 0x7f69e81394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:25.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69edee8640 1 -- 172.21.15.98:0/3907472472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69e815c670 msgr2=0x7f69e80b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:25.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69edee8640 1 --2- 172.21.15.98:0/3907472472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69e815c670 0x7f69e80b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:25.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69edee8640 1 -- 172.21.15.98:0/3907472472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e81399f0 con 0x7f69e814aab0 2024-09-10T20:59:25.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.366+0000 7f69edee8640 1 --2- 172.21.15.98:0/3907472472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69e814aab0 0x7f69e80b0de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f69e80cf9b0 tx=0x7f69e00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:25.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.368+0000 7f69e5ffb640 1 -- 172.21.15.98:0/3907472472 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0015020 con 0x7f69e814aab0 2024-09-10T20:59:25.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.368+0000 7f69ef94b640 1 -- 172.21.15.98:0/3907472472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e8158400 con 0x7f69e814aab0 2024-09-10T20:59:25.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.368+0000 7f69e5ffb640 1 -- 172.21.15.98:0/3907472472 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69e0011630 con 0x7f69e814aab0 2024-09-10T20:59:25.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.368+0000 7f69e5ffb640 1 -- 172.21.15.98:0/3907472472 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e00119e0 con 0x7f69e814aab0 2024-09-10T20:59:25.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.368+0000 7f69ef94b640 1 -- 172.21.15.98:0/3907472472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e815cea0 con 0x7f69e814aab0 2024-09-10T20:59:25.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.369+0000 7f69ef94b640 1 -- 172.21.15.98:0/3907472472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e80b4a30 con 0x7f69e814aab0 2024-09-10T20:59:25.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.370+0000 7f69e5ffb640 1 -- 172.21.15.98:0/3907472472 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f69e0011b80 con 0x7f69e814aab0 2024-09-10T20:59:25.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.370+0000 7f69e5ffb640 1 --2- 172.21.15.98:0/3907472472 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f69d4077fa0 0x7f69d407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:25.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.371+0000 7f69e5ffb640 1 -- 172.21.15.98:0/3907472472 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f69e009ab40 con 0x7f69e814aab0 2024-09-10T20:59:25.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.371+0000 7f69ed6e7640 1 --2- 172.21.15.98:0/3907472472 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f69d4077fa0 0x7f69d407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:25.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.371+0000 7f69ed6e7640 1 --2- 172.21.15.98:0/3907472472 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f69d4077fa0 0x7f69d407a460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f69d8005fd0 tx=0x7f69d8009530 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:25.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.376+0000 7f69e5ffb640 1 -- 172.21.15.98:0/3907472472 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f69e001f020 con 0x7f69e814aab0 2024-09-10T20:59:25.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.564+0000 7f69ef94b640 1 -- 172.21.15.98:0/3907472472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f69e80cc6a0 con 0x7f69e814aab0 2024-09-10T20:59:25.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.565+0000 7f69e5ffb640 1 -- 172.21.15.98:0/3907472472 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f69e81399f0 con 0x7f69e814aab0 2024-09-10T20:59:25.569 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-10T20:59:25.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.567+0000 7f69ef94b640 1 -- 172.21.15.98:0/3907472472 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f69d4077fa0 msgr2=0x7f69d407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:25.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.567+0000 7f69ef94b640 1 --2- 172.21.15.98:0/3907472472 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f69d4077fa0 0x7f69d407a460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f69d8005fd0 tx=0x7f69d8009530 comp rx=0 tx=0).stop 2024-09-10T20:59:25.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.567+0000 7f69ef94b640 1 -- 172.21.15.98:0/3907472472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69e814aab0 msgr2=0x7f69e80b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:25.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.567+0000 7f69ef94b640 1 --2- 172.21.15.98:0/3907472472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69e814aab0 0x7f69e80b0de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f69e80cf9b0 tx=0x7f69e00039c0 comp rx=0 tx=0).stop 2024-09-10T20:59:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.567+0000 7f69ef94b640 1 -- 172.21.15.98:0/3907472472 shutdown_connections 2024-09-10T20:59:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.567+0000 7f69ef94b640 1 --2- 172.21.15.98:0/3907472472 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f69d4077fa0 0x7f69d407a460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.567+0000 7f69ef94b640 1 --2- 172.21.15.98:0/3907472472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69e814aab0 0x7f69e80b0de0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.567+0000 7f69ef94b640 1 --2- 172.21.15.98:0/3907472472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69e815c670 0x7f69e80b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.567+0000 7f69ef94b640 1 --2- 172.21.15.98:0/3907472472 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69d0004800 0x7f69e81394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.567+0000 7f69ef94b640 1 -- 172.21.15.98:0/3907472472 >> 172.21.15.98:0/3907472472 conn(0x7f69e80b1760 msgr2=0x7f69e806f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.567+0000 7f69ef94b640 1 -- 172.21.15.98:0/3907472472 shutdown_connections 2024-09-10T20:59:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.567+0000 7f69ef94b640 1 -- 172.21.15.98:0/3907472472 wait complete. 2024-09-10T20:59:25.581 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T20:59:25.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:25.582 DEBUG:teuthology.orchestra.run.smithi098:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T20:59:25.608 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T20:59:25.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:25.608 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T20:59:25.664 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:59:25.664 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:25.684 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:59:25.684 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:25.742 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:59:25.742 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:25.801 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T20:59:25.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.899+0000 7f6518e7f640 1 -- 172.21.15.98:0/2724166355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64fc003a00 msgr2=0x7f64fc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:25.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.899+0000 7f6518e7f640 1 --2- 172.21.15.98:0/2724166355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64fc003a00 0x7f64fc003e20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6504009ed0 tx=0x7f64fc005830 comp rx=0 tx=0).stop 2024-09-10T20:59:25.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.900+0000 7f6512d9d640 1 -- 172.21.15.98:0/2724166355 reap_dead start 2024-09-10T20:59:25.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.900+0000 7f6518e7f640 1 -- 172.21.15.98:0/2724166355 shutdown_connections 2024-09-10T20:59:25.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.900+0000 7f6518e7f640 1 -- 172.21.15.98:0/2724166355 >> 172.21.15.98:0/2724166355 conn(0x7f65140d0c00 msgr2=0x7f65140d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:25.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.900+0000 7f6518e7f640 1 -- 172.21.15.98:0/2724166355 shutdown_connections 2024-09-10T20:59:25.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.900+0000 7f6518e7f640 1 -- 172.21.15.98:0/2724166355 wait complete. 2024-09-10T20:59:25.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.901+0000 7f6518e7f640 1 Processor -- start 2024-09-10T20:59:25.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.901+0000 7f6518e7f640 1 -- start start 2024-09-10T20:59:25.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6518e7f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64fc003a00 0x7f65141f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:25.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6518e7f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6514152c10 0x7f65141f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:25.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6518e7f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65141f5810 0x7f65141f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:25.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6518e7f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65140b4180 con 0x7f6514152c10 2024-09-10T20:59:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6518e7f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f65140b4000 con 0x7f64fc003a00 2024-09-10T20:59:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6518e7f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f65140b4300 con 0x7f65141f5810 2024-09-10T20:59:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6511d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6514152c10 0x7f65141f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6511d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6514152c10 0x7f65141f52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39460/0 (socket says 172.21.15.98:39460) 2024-09-10T20:59:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6511d9b640 1 -- 172.21.15.98:0/1726941400 learned_addr learned my addr 172.21.15.98:0/1726941400 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f651259c640 1 --2- 172.21.15.98:0/1726941400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64fc003a00 0x7f65141f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6512d9d640 1 --2- 172.21.15.98:0/1726941400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65141f5810 0x7f65141f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6511d9b640 1 -- 172.21.15.98:0/1726941400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65141f5810 msgr2=0x7f65141f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6511d9b640 1 --2- 172.21.15.98:0/1726941400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65141f5810 0x7f65141f3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6511d9b640 1 -- 172.21.15.98:0/1726941400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64fc003a00 msgr2=0x7f65141f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6511d9b640 1 --2- 172.21.15.98:0/1726941400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64fc003a00 0x7f65141f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:25.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6511d9b640 1 -- 172.21.15.98:0/1726941400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65141531d0 con 0x7f6514152c10 2024-09-10T20:59:25.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6511d9b640 1 --2- 172.21.15.98:0/1726941400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6514152c10 0x7f65141f52d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f650800e6d0 tx=0x7f650800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:25.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.902+0000 7f6502ffd640 1 -- 172.21.15.98:0/1726941400 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6508015020 con 0x7f6514152c10 2024-09-10T20:59:25.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.903+0000 7f6502ffd640 1 -- 172.21.15.98:0/1726941400 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65080035f0 con 0x7f6514152c10 2024-09-10T20:59:25.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.903+0000 7f6502ffd640 1 -- 172.21.15.98:0/1726941400 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6508003a40 con 0x7f6514152c10 2024-09-10T20:59:25.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.903+0000 7f6518e7f640 1 -- 172.21.15.98:0/1726941400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65141534b0 con 0x7f6514152c10 2024-09-10T20:59:25.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.903+0000 7f6518e7f640 1 -- 172.21.15.98:0/1726941400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6514153a70 con 0x7f6514152c10 2024-09-10T20:59:25.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.904+0000 7f6518e7f640 1 -- 172.21.15.98:0/1726941400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f651415eab0 con 0x7f6514152c10 2024-09-10T20:59:25.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.904+0000 7f6502ffd640 1 -- 172.21.15.98:0/1726941400 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6508003be0 con 0x7f6514152c10 2024-09-10T20:59:25.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.905+0000 7f6502ffd640 1 --2- 172.21.15.98:0/1726941400 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f64f4077f70 0x7f64f407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:25.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.905+0000 7f6502ffd640 1 -- 172.21.15.98:0/1726941400 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f650809a710 con 0x7f6514152c10 2024-09-10T20:59:25.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.905+0000 7f651259c640 1 --2- 172.21.15.98:0/1726941400 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f64f4077f70 0x7f64f407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:25.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.905+0000 7f651259c640 1 --2- 172.21.15.98:0/1726941400 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f64f4077f70 0x7f64f407a430 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6504035000 tx=0x7f650402d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:25.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:25.909+0000 7f6502ffd640 1 -- 172.21.15.98:0/1726941400 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f650801f030 con 0x7f6514152c10 2024-09-10T20:59:26.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.092+0000 7f6518e7f640 1 -- 172.21.15.98:0/1726941400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6514151fe0 con 0x7f6514152c10 2024-09-10T20:59:26.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.092+0000 7f6502ffd640 1 -- 172.21.15.98:0/1726941400 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f6508062ec0 con 0x7f6514152c10 2024-09-10T20:59:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-10T20:59:26.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.094+0000 7f6518e7f640 1 -- 172.21.15.98:0/1726941400 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f64f4077f70 msgr2=0x7f64f407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:26.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.094+0000 7f6518e7f640 1 --2- 172.21.15.98:0/1726941400 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f64f4077f70 0x7f64f407a430 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6504035000 tx=0x7f650402d040 comp rx=0 tx=0).stop 2024-09-10T20:59:26.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.094+0000 7f6518e7f640 1 -- 172.21.15.98:0/1726941400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6514152c10 msgr2=0x7f65141f52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:26.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.094+0000 7f6518e7f640 1 --2- 172.21.15.98:0/1726941400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6514152c10 0x7f65141f52d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f650800e6d0 tx=0x7f650800ead0 comp rx=0 tx=0).stop 2024-09-10T20:59:26.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.094+0000 7f6518e7f640 1 -- 172.21.15.98:0/1726941400 shutdown_connections 2024-09-10T20:59:26.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.094+0000 7f6518e7f640 1 --2- 172.21.15.98:0/1726941400 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f64f4077f70 0x7f64f407a430 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:26.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.094+0000 7f6518e7f640 1 --2- 172.21.15.98:0/1726941400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f65141f5810 0x7f65141f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:26.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.094+0000 7f6518e7f640 1 --2- 172.21.15.98:0/1726941400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6514152c10 0x7f65141f52d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:26.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.094+0000 7f6518e7f640 1 --2- 172.21.15.98:0/1726941400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64fc003a00 0x7f65141f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:26.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.094+0000 7f6518e7f640 1 -- 172.21.15.98:0/1726941400 >> 172.21.15.98:0/1726941400 conn(0x7f65140d0c00 msgr2=0x7f651415dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:26.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.094+0000 7f6518e7f640 1 -- 172.21.15.98:0/1726941400 shutdown_connections 2024-09-10T20:59:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.094+0000 7f6518e7f640 1 -- 172.21.15.98:0/1726941400 wait complete. 2024-09-10T20:59:26.109 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T20:59:26.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:26.109 DEBUG:teuthology.orchestra.run.smithi190:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-10T20:59:26.141 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2024-09-10T20:59:26.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:26.141 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T20:59:26.205 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:59:26.205 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:26.221 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:59:26.221 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:26.278 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:59:26.278 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:26.334 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T20:59:26.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.472+0000 7fbdbc40c640 1 -- 172.21.15.98:0/302536944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb415c470 msgr2=0x7fbdb415c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:26.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.472+0000 7fbdbc40c640 1 --2- 172.21.15.98:0/302536944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb415c470 0x7fbdb415c8e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4076000 tx=0x7fbda402a560 comp rx=0 tx=0).stop 2024-09-10T20:59:26.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.472+0000 7fbdbc40c640 1 -- 172.21.15.98:0/302536944 shutdown_connections 2024-09-10T20:59:26.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.472+0000 7fbdbc40c640 1 --2- 172.21.15.98:0/302536944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb415c470 0x7fbdb415c8e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:26.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.472+0000 7fbdbc40c640 1 --2- 172.21.15.98:0/302536944 >> v2:172.21.15.190:3300/0 conn(0x7fbdb4073780 0x7fbdb4073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:26.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.472+0000 7fbdbc40c640 1 -- 172.21.15.98:0/302536944 >> 172.21.15.98:0/302536944 conn(0x7fbdb40b1760 msgr2=0x7fbdb40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:26.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.472+0000 7fbdbc40c640 1 -- 172.21.15.98:0/302536944 shutdown_connections 2024-09-10T20:59:26.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.472+0000 7fbdbc40c640 1 -- 172.21.15.98:0/302536944 wait complete. 2024-09-10T20:59:26.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.473+0000 7fbdbc40c640 1 Processor -- start 2024-09-10T20:59:26.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.473+0000 7fbdbc40c640 1 -- start start 2024-09-10T20:59:26.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdbc40c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbdb4072560 0x7fbdb4151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdbc40c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdb4073780 0x7fbdb41f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdbc40c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb415c470 0x7fbdb41f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdbc40c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb415cfa0 con 0x7fbdb415c470 2024-09-10T20:59:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdbc40c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb415ce20 con 0x7fbdb4073780 2024-09-10T20:59:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdbc40c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb4070150 con 0x7fbdb4072560 2024-09-10T20:59:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdba9a9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb415c470 0x7fbdb41f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdba9a9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb415c470 0x7fbdb41f3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39468/0 (socket says 172.21.15.98:39468) 2024-09-10T20:59:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdba9a9640 1 -- 172.21.15.98:0/2725144118 learned_addr learned my addr 172.21.15.98:0/2725144118 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdb99a7640 1 --2- 172.21.15.98:0/2725144118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdb4073780 0x7fbdb41f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdba1a8640 1 --2- 172.21.15.98:0/2725144118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbdb4072560 0x7fbdb4151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdba9a9640 1 -- 172.21.15.98:0/2725144118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbdb4072560 msgr2=0x7fbdb4151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdba9a9640 1 --2- 172.21.15.98:0/2725144118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbdb4072560 0x7fbdb4151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdba9a9640 1 -- 172.21.15.98:0/2725144118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdb4073780 msgr2=0x7fbdb41f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdba9a9640 1 --2- 172.21.15.98:0/2725144118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdb4073780 0x7fbdb41f51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdba9a9640 1 -- 172.21.15.98:0/2725144118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb41f4100 con 0x7fbdb415c470 2024-09-10T20:59:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdba9a9640 1 --2- 172.21.15.98:0/2725144118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb415c470 0x7fbdb41f3640 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00c7b0 tx=0x7fbdac00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdaaffd640 1 -- 172.21.15.98:0/2725144118 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac004b60 con 0x7fbdb415c470 2024-09-10T20:59:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdaaffd640 1 -- 172.21.15.98:0/2725144118 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdac004df0 con 0x7fbdb415c470 2024-09-10T20:59:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdbc40c640 1 -- 172.21.15.98:0/2725144118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb41f43e0 con 0x7fbdb415c470 2024-09-10T20:59:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.474+0000 7fbdaaffd640 1 -- 172.21.15.98:0/2725144118 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00aa90 con 0x7fbdb415c470 2024-09-10T20:59:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.475+0000 7fbdbc40c640 1 -- 172.21.15.98:0/2725144118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb41f4c80 con 0x7fbdb415c470 2024-09-10T20:59:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.476+0000 7fbdbc40c640 1 -- 172.21.15.98:0/2725144118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb4073250 con 0x7fbdb415c470 2024-09-10T20:59:26.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.476+0000 7fbdaaffd640 1 -- 172.21.15.98:0/2725144118 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbdac00ac30 con 0x7fbdb415c470 2024-09-10T20:59:26.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.477+0000 7fbdaaffd640 1 --2- 172.21.15.98:0/2725144118 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbd98077f70 0x7fbd9807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:26.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.477+0000 7fbdaaffd640 1 -- 172.21.15.98:0/2725144118 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbdac09a1f0 con 0x7fbdb415c470 2024-09-10T20:59:26.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.477+0000 7fbdba1a8640 1 --2- 172.21.15.98:0/2725144118 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbd98077f70 0x7fbd9807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:26.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.477+0000 7fbdba1a8640 1 --2- 172.21.15.98:0/2725144118 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbd98077f70 0x7fbd9807a430 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0005fd0 tx=0x7fbdb000b390 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:26.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.481+0000 7fbdaaffd640 1 -- 172.21.15.98:0/2725144118 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbdac020070 con 0x7fbdb415c470 2024-09-10T20:59:26.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.665+0000 7fbdbc40c640 1 -- 172.21.15.98:0/2725144118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbdb40cd6d0 con 0x7fbdb415c470 2024-09-10T20:59:26.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.665+0000 7fbdaaffd640 1 -- 172.21.15.98:0/2725144118 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fbdac062360 con 0x7fbdb415c470 2024-09-10T20:59:26.669 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-10T20:59:26.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.667+0000 7fbdbc40c640 1 -- 172.21.15.98:0/2725144118 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbd98077f70 msgr2=0x7fbd9807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:26.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.667+0000 7fbdbc40c640 1 --2- 172.21.15.98:0/2725144118 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbd98077f70 0x7fbd9807a430 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0005fd0 tx=0x7fbdb000b390 comp rx=0 tx=0).stop 2024-09-10T20:59:26.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.667+0000 7fbdbc40c640 1 -- 172.21.15.98:0/2725144118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb415c470 msgr2=0x7fbdb41f3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:26.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.667+0000 7fbdbc40c640 1 --2- 172.21.15.98:0/2725144118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb415c470 0x7fbdb41f3640 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00c7b0 tx=0x7fbdac00cc70 comp rx=0 tx=0).stop 2024-09-10T20:59:26.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.667+0000 7fbdbc40c640 1 -- 172.21.15.98:0/2725144118 shutdown_connections 2024-09-10T20:59:26.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.667+0000 7fbdbc40c640 1 --2- 172.21.15.98:0/2725144118 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fbd98077f70 0x7fbd9807a430 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:26.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.667+0000 7fbdbc40c640 1 --2- 172.21.15.98:0/2725144118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb415c470 0x7fbdb41f3640 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:26.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.667+0000 7fbdbc40c640 1 --2- 172.21.15.98:0/2725144118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbdb4073780 0x7fbdb41f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:26.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.667+0000 7fbdbc40c640 1 --2- 172.21.15.98:0/2725144118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbdb4072560 0x7fbdb4151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:26.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.667+0000 7fbdbc40c640 1 -- 172.21.15.98:0/2725144118 >> 172.21.15.98:0/2725144118 conn(0x7fbdb40b1760 msgr2=0x7fbdb40ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:26.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.667+0000 7fbdbc40c640 1 -- 172.21.15.98:0/2725144118 shutdown_connections 2024-09-10T20:59:26.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:26.667+0000 7fbdbc40c640 1 -- 172.21.15.98:0/2725144118 wait complete. 2024-09-10T20:59:26.681 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-10T20:59:26.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:26.682 DEBUG:teuthology.orchestra.run.smithi098:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-10T20:59:26.707 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T20:59:26.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:26.708 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T20:59:26.763 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:59:26.763 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:26.784 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:59:26.784 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:26.843 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T20:59:26.843 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:26.902 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T20:59:27.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.000+0000 7f5e4a990640 1 -- 172.21.15.98:0/627822475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e440cf210 msgr2=0x7f5e4415c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.000+0000 7f5e4a990640 1 --2- 172.21.15.98:0/627822475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e440cf210 0x7f5e4415c550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c005fd0 tx=0x7f5e3c02aa10 comp rx=0 tx=0).stop 2024-09-10T20:59:27.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.000+0000 7f5e4a990640 1 -- 172.21.15.98:0/627822475 shutdown_connections 2024-09-10T20:59:27.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.000+0000 7f5e4a990640 1 --2- 172.21.15.98:0/627822475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e440cf210 0x7f5e4415c550 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.000+0000 7f5e4a990640 1 -- 172.21.15.98:0/627822475 >> 172.21.15.98:0/627822475 conn(0x7f5e440b1760 msgr2=0x7f5e440b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:27.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.000+0000 7f5e4a990640 1 -- 172.21.15.98:0/627822475 shutdown_connections 2024-09-10T20:59:27.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.000+0000 7f5e4a990640 1 -- 172.21.15.98:0/627822475 wait complete. 2024-09-10T20:59:27.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.001+0000 7f5e4a990640 1 Processor -- start 2024-09-10T20:59:27.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.001+0000 7f5e4a990640 1 -- start start 2024-09-10T20:59:27.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e4a990640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e44072680 0x7f5e44159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:27.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e4a990640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e440cf210 0x7f5e441e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:27.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e4a990640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e44144780 0x7f5e441e6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:27.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e4a990640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4415cc10 con 0x7f5e440cf210 2024-09-10T20:59:27.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e4a990640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4415ca90 con 0x7f5e44072680 2024-09-10T20:59:27.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e4a990640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5e4415cd90 con 0x7f5e44144780 2024-09-10T20:59:27.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e3bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e440cf210 0x7f5e441e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:27.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e3bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e440cf210 0x7f5e441e7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39492/0 (socket says 172.21.15.98:39492) 2024-09-10T20:59:27.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e3bfff640 1 -- 172.21.15.98:0/3657616264 learned_addr learned my addr 172.21.15.98:0/3657616264 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:27.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e43fff640 1 --2- 172.21.15.98:0/3657616264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e44072680 0x7f5e44159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:27.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e48f2d640 1 --2- 172.21.15.98:0/3657616264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e44144780 0x7f5e441e6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:27.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e3bfff640 1 -- 172.21.15.98:0/3657616264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e44144780 msgr2=0x7f5e441e6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e3bfff640 1 --2- 172.21.15.98:0/3657616264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e44144780 0x7f5e441e6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e3bfff640 1 -- 172.21.15.98:0/3657616264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e44072680 msgr2=0x7f5e44159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e3bfff640 1 --2- 172.21.15.98:0/3657616264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e44072680 0x7f5e44159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e3bfff640 1 -- 172.21.15.98:0/3657616264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e441e6b70 con 0x7f5e440cf210 2024-09-10T20:59:27.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e3bfff640 1 --2- 172.21.15.98:0/3657616264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e440cf210 0x7f5e441e7c10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00e760 tx=0x7f5e2c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:27.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e417fa640 1 -- 172.21.15.98:0/3657616264 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c0041f0 con 0x7f5e440cf210 2024-09-10T20:59:27.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e417fa640 1 -- 172.21.15.98:0/3657616264 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e2c011470 con 0x7f5e440cf210 2024-09-10T20:59:27.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e417fa640 1 -- 172.21.15.98:0/3657616264 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c011900 con 0x7f5e440cf210 2024-09-10T20:59:27.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.002+0000 7f5e4a990640 1 -- 172.21.15.98:0/3657616264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e441e6e50 con 0x7f5e440cf210 2024-09-10T20:59:27.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.003+0000 7f5e4a990640 1 -- 172.21.15.98:0/3657616264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e441e7410 con 0x7f5e440cf210 2024-09-10T20:59:27.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.003+0000 7f5e4a990640 1 -- 172.21.15.98:0/3657616264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e4415c550 con 0x7f5e440cf210 2024-09-10T20:59:27.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.004+0000 7f5e417fa640 1 -- 172.21.15.98:0/3657616264 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5e2c00f040 con 0x7f5e440cf210 2024-09-10T20:59:27.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.005+0000 7f5e417fa640 1 --2- 172.21.15.98:0/3657616264 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5e24077f70 0x7f5e2407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:27.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.005+0000 7f5e417fa640 1 -- 172.21.15.98:0/3657616264 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5e2c09c510 con 0x7f5e440cf210 2024-09-10T20:59:27.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.005+0000 7f5e43fff640 1 --2- 172.21.15.98:0/3657616264 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5e24077f70 0x7f5e2407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:27.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.005+0000 7f5e43fff640 1 --2- 172.21.15.98:0/3657616264 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5e24077f70 0x7f5e2407a430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5e34001720 tx=0x7f5e34009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:27.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.009+0000 7f5e417fa640 1 -- 172.21.15.98:0/3657616264 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5e2c016030 con 0x7f5e440cf210 2024-09-10T20:59:27.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.194+0000 7f5e4a990640 1 -- 172.21.15.98:0/3657616264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5e44143f30 con 0x7f5e440cf210 2024-09-10T20:59:27.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.195+0000 7f5e417fa640 1 -- 172.21.15.98:0/3657616264 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f5e2c064d60 con 0x7f5e440cf210 2024-09-10T20:59:27.199 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-10T20:59:27.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.197+0000 7f5e4a990640 1 -- 172.21.15.98:0/3657616264 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5e24077f70 msgr2=0x7f5e2407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.197+0000 7f5e4a990640 1 --2- 172.21.15.98:0/3657616264 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5e24077f70 0x7f5e2407a430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5e34001720 tx=0x7f5e34009040 comp rx=0 tx=0).stop 2024-09-10T20:59:27.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.197+0000 7f5e4a990640 1 -- 172.21.15.98:0/3657616264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e440cf210 msgr2=0x7f5e441e7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.197+0000 7f5e4a990640 1 --2- 172.21.15.98:0/3657616264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e440cf210 0x7f5e441e7c10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00e760 tx=0x7f5e2c00ec20 comp rx=0 tx=0).stop 2024-09-10T20:59:27.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.197+0000 7f5e4a990640 1 -- 172.21.15.98:0/3657616264 shutdown_connections 2024-09-10T20:59:27.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.197+0000 7f5e4a990640 1 --2- 172.21.15.98:0/3657616264 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5e24077f70 0x7f5e2407a430 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.197+0000 7f5e4a990640 1 --2- 172.21.15.98:0/3657616264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e44144780 0x7f5e441e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.197+0000 7f5e4a990640 1 --2- 172.21.15.98:0/3657616264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e440cf210 0x7f5e441e7c10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.197+0000 7f5e4a990640 1 --2- 172.21.15.98:0/3657616264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e44072680 0x7f5e44159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.197+0000 7f5e4a990640 1 -- 172.21.15.98:0/3657616264 >> 172.21.15.98:0/3657616264 conn(0x7f5e440b1760 msgr2=0x7f5e440b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:27.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.197+0000 7f5e4a990640 1 -- 172.21.15.98:0/3657616264 shutdown_connections 2024-09-10T20:59:27.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.197+0000 7f5e4a990640 1 -- 172.21.15.98:0/3657616264 wait complete. 2024-09-10T20:59:27.211 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-10T20:59:27.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:27.212 DEBUG:teuthology.orchestra.run.smithi190:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-10T20:59:27.245 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2024-09-10T20:59:27.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:27.246 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T20:59:27.304 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T20:59:27.304 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:59:27.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.406+0000 7ffaddbc6640 1 -- 172.21.15.98:0/622262410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad8072680 msgr2=0x7ffad80ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.406+0000 7ffaddbc6640 1 --2- 172.21.15.98:0/622262410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad8072680 0x7ffad80ceb00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009830 tx=0x7ffac002a530 comp rx=0 tx=0).stop 2024-09-10T20:59:27.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.406+0000 7ffaddbc6640 1 -- 172.21.15.98:0/622262410 shutdown_connections 2024-09-10T20:59:27.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.406+0000 7ffaddbc6640 1 --2- 172.21.15.98:0/622262410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad8072680 0x7ffad80ceb00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.406+0000 7ffaddbc6640 1 -- 172.21.15.98:0/622262410 >> 172.21.15.98:0/622262410 conn(0x7ffad80b1760 msgr2=0x7ffad80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:27.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.406+0000 7ffaddbc6640 1 -- 172.21.15.98:0/622262410 shutdown_connections 2024-09-10T20:59:27.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.406+0000 7ffaddbc6640 1 -- 172.21.15.98:0/622262410 wait complete. 2024-09-10T20:59:27.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.407+0000 7ffaddbc6640 1 Processor -- start 2024-09-10T20:59:27.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.408+0000 7ffaddbc6640 1 -- start start 2024-09-10T20:59:27.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.408+0000 7ffaddbc6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad8072680 0x7ffad813ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:27.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.408+0000 7ffaddbc6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad8072ad0 0x7ffad81397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:27.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.408+0000 7ffaddbc6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad8144780 0x7ffad81e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:27.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.408+0000 7ffaddbc6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffad815cc60 con 0x7ffad8072680 2024-09-10T20:59:27.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.408+0000 7ffaddbc6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffad80cf390 con 0x7ffad8144780 2024-09-10T20:59:27.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.408+0000 7ffaddbc6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffad815cde0 con 0x7ffad8072ad0 2024-09-10T20:59:27.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.408+0000 7ffad77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad8072680 0x7ffad813ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:27.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffad77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad8072680 0x7ffad813ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39512/0 (socket says 172.21.15.98:39512) 2024-09-10T20:59:27.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffad77fe640 1 -- 172.21.15.98:0/90844034 learned_addr learned my addr 172.21.15.98:0/90844034 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:27.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffad7fff640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad8144780 0x7ffad81e8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:27.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffaceffd640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad8072ad0 0x7ffad81397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:27.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffad77fe640 1 -- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad8072ad0 msgr2=0x7ffad81397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffad77fe640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad8072ad0 0x7ffad81397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffad77fe640 1 -- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad8144780 msgr2=0x7ffad81e8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffad77fe640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad8144780 0x7ffad81e8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffad77fe640 1 -- 172.21.15.98:0/90844034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad813a020 con 0x7ffad8072680 2024-09-10T20:59:27.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffaceffd640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad8072ad0 0x7ffad81397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:27.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffad77fe640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad8072680 0x7ffad813ade0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffac800c3a0 tx=0x7ffac800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:27.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffad4ff9640 1 -- 172.21.15.98:0/90844034 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8017070 con 0x7ffad8072680 2024-09-10T20:59:27.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffad4ff9640 1 -- 172.21.15.98:0/90844034 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffac80053c0 con 0x7ffad8072680 2024-09-10T20:59:27.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffaddbc6640 1 -- 172.21.15.98:0/90844034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad813a300 con 0x7ffad8072680 2024-09-10T20:59:27.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffad4ff9640 1 -- 172.21.15.98:0/90844034 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8003850 con 0x7ffad8072680 2024-09-10T20:59:27.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.409+0000 7ffaddbc6640 1 -- 172.21.15.98:0/90844034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad813a8c0 con 0x7ffad8072680 2024-09-10T20:59:27.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.410+0000 7ffaddbc6640 1 -- 172.21.15.98:0/90844034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad80cf810 con 0x7ffad8072680 2024-09-10T20:59:27.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.411+0000 7ffad4ff9640 1 -- 172.21.15.98:0/90844034 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ffac80039f0 con 0x7ffad8072680 2024-09-10T20:59:27.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.411+0000 7ffad4ff9640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffab8077f70 0x7ffab807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:27.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.412+0000 7ffad4ff9640 1 -- 172.21.15.98:0/90844034 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6714+0+0 (secure 0 0 0) 0x7ffac8099680 con 0x7ffad8072680 2024-09-10T20:59:27.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.412+0000 7ffaceffd640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffab8077f70 0x7ffab807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:27.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.412+0000 7ffaceffd640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffab8077f70 0x7ffab807a430 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffac0034000 tx=0x7ffac0004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:27.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.416+0000 7ffad4ff9640 1 -- 172.21.15.98:0/90844034 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ffac80617d0 con 0x7ffad8072680 2024-09-10T20:59:27.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.626+0000 7ffaddbc6640 1 -- 172.21.15.98:0/90844034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffad80b4d40 con 0x7ffad8072680 2024-09-10T20:59:27.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.626+0000 7ffad4ff9640 1 -- 172.21.15.98:0/90844034 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7ffac8066200 con 0x7ffad8072680 2024-09-10T20:59:27.631 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:27.631 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"btime":"2024-09-10T20:59:06:294252+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4277,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/1954552281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1954552281},{"type":"v1","addr":"172.21.15.190:6836","nonce":1954552281}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/8997965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":8997965},{"type":"v1","addr":"172.21.15.190:6837","nonce":8997965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1984550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1984550306},{"type":"v1","addr":"172.21.15.98:6833","nonce":1984550306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:58:46.560479+0000","modified":"2024-09-10T20:59:05.635381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3141703317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3141703317},{"type":"v1","addr":"172.21.15.98:6835","nonce":3141703317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T20:59:27.631 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2024-09-10T20:59:27.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.628+0000 7ffaddbc6640 1 -- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffab8077f70 msgr2=0x7ffab807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.628+0000 7ffaddbc6640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffab8077f70 0x7ffab807a430 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffac0034000 tx=0x7ffac0004730 comp rx=0 tx=0).stop 2024-09-10T20:59:27.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.628+0000 7ffaddbc6640 1 -- 172.21.15.98:0/90844034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad8072680 msgr2=0x7ffad813ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.628+0000 7ffaddbc6640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad8072680 0x7ffad813ade0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffac800c3a0 tx=0x7ffac800c860 comp rx=0 tx=0).stop 2024-09-10T20:59:27.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.629+0000 7ffaddbc6640 1 -- 172.21.15.98:0/90844034 shutdown_connections 2024-09-10T20:59:27.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.629+0000 7ffaddbc6640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffab8077f70 0x7ffab807a430 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.629+0000 7ffaddbc6640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad8144780 0x7ffad81e8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.629+0000 7ffaddbc6640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad8072ad0 0x7ffad81397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.629+0000 7ffaddbc6640 1 --2- 172.21.15.98:0/90844034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad8072680 0x7ffad813ade0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.629+0000 7ffaddbc6640 1 -- 172.21.15.98:0/90844034 >> 172.21.15.98:0/90844034 conn(0x7ffad80b1760 msgr2=0x7ffad80cffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.629+0000 7ffaddbc6640 1 -- 172.21.15.98:0/90844034 shutdown_connections 2024-09-10T20:59:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.629+0000 7ffaddbc6640 1 -- 172.21.15.98:0/90844034 wait complete. 2024-09-10T20:59:27.643 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:59:27.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.741+0000 7f590fdec640 1 -- 172.21.15.98:0/3017168251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58f0004d40 msgr2=0x7f58f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.741+0000 7f590fdec640 1 --2- 172.21.15.98:0/3017168251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58f0004d40 0x7f58f00051c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f59080af6a0 tx=0x7f58f0005830 comp rx=0 tx=0).stop 2024-09-10T20:59:27.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.741+0000 7f590e389640 1 -- 172.21.15.98:0/3017168251 reap_dead start 2024-09-10T20:59:27.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.741+0000 7f590fdec640 1 -- 172.21.15.98:0/3017168251 shutdown_connections 2024-09-10T20:59:27.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.741+0000 7f590fdec640 1 -- 172.21.15.98:0/3017168251 >> 172.21.15.98:0/3017168251 conn(0x7f59080b1760 msgr2=0x7f59080b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:27.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.741+0000 7f590fdec640 1 -- 172.21.15.98:0/3017168251 shutdown_connections 2024-09-10T20:59:27.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.741+0000 7f590fdec640 1 -- 172.21.15.98:0/3017168251 wait complete. 2024-09-10T20:59:27.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.742+0000 7f590fdec640 1 Processor -- start 2024-09-10T20:59:27.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.742+0000 7f590fdec640 1 -- start start 2024-09-10T20:59:27.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590fdec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58f0004d40 0x7f590815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590fdec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59081406c0 0x7f5908070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590fdec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f590806f330 0x7f590806f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590fdec640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f59080cef90 con 0x7f590806f330 2024-09-10T20:59:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590fdec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f590815d0b0 con 0x7f58f0004d40 2024-09-10T20:59:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590fdec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f59080cf110 con 0x7f59081406c0 2024-09-10T20:59:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590e389640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f590806f330 0x7f590806f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590e389640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f590806f330 0x7f590806f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39530/0 (socket says 172.21.15.98:39530) 2024-09-10T20:59:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590e389640 1 -- 172.21.15.98:0/1937608446 learned_addr learned my addr 172.21.15.98:0/1937608446 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590db88640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58f0004d40 0x7f590815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590d387640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59081406c0 0x7f5908070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590e389640 1 -- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59081406c0 msgr2=0x7f5908070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590e389640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59081406c0 0x7f5908070090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590e389640 1 -- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58f0004d40 msgr2=0x7f590815a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590e389640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58f0004d40 0x7f590815a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590e389640 1 -- 172.21.15.98:0/1937608446 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59080c49d0 con 0x7f590806f330 2024-09-10T20:59:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590db88640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58f0004d40 0x7f590815a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.743+0000 7f590e389640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f590806f330 0x7f590806f7e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f58f802b470 tx=0x7f58f8004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.744+0000 7f59067fc640 1 -- 172.21.15.98:0/1937608446 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f802b7c0 con 0x7f590806f330 2024-09-10T20:59:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.744+0000 7f59067fc640 1 -- 172.21.15.98:0/1937608446 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58f802b960 con 0x7f590806f330 2024-09-10T20:59:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.744+0000 7f590fdec640 1 -- 172.21.15.98:0/1937608446 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59080c4c70 con 0x7f590806f330 2024-09-10T20:59:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.744+0000 7f59067fc640 1 -- 172.21.15.98:0/1937608446 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f802bd30 con 0x7f590806f330 2024-09-10T20:59:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.744+0000 7f590fdec640 1 -- 172.21.15.98:0/1937608446 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59080c5230 con 0x7f590806f330 2024-09-10T20:59:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.745+0000 7f590fdec640 1 -- 172.21.15.98:0/1937608446 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5908072aa0 con 0x7f590806f330 2024-09-10T20:59:27.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.745+0000 7f59067fc640 1 -- 172.21.15.98:0/1937608446 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f58f80060a0 con 0x7f590806f330 2024-09-10T20:59:27.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.746+0000 7f59067fc640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f58f4077f70 0x7f58f407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:27.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.746+0000 7f59067fc640 1 -- 172.21.15.98:0/1937608446 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6714+0+0 (secure 0 0 0) 0x7f58f80b9b30 con 0x7f590806f330 2024-09-10T20:59:27.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.746+0000 7f590db88640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f58f4077f70 0x7f58f407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:27.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.746+0000 7f590db88640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f58f4077f70 0x7f58f407a430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f59000045e0 tx=0x7f590000b260 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:27.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.750+0000 7f59067fc640 1 -- 172.21.15.98:0/1937608446 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f58f80821b0 con 0x7f590806f330 2024-09-10T20:59:27.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.958+0000 7f590fdec640 1 -- 172.21.15.98:0/1937608446 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59080cf810 con 0x7f590806f330 2024-09-10T20:59:27.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.958+0000 7f59067fc640 1 -- 172.21.15.98:0/1937608446 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7f58f8081ab0 con 0x7f590806f330 2024-09-10T20:59:27.963 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:27.963 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"btime":"2024-09-10T20:59:06:294252+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4277,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/1954552281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1954552281},{"type":"v1","addr":"172.21.15.190:6836","nonce":1954552281}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/8997965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":8997965},{"type":"v1","addr":"172.21.15.190:6837","nonce":8997965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1984550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1984550306},{"type":"v1","addr":"172.21.15.98:6833","nonce":1984550306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:58:46.560479+0000","modified":"2024-09-10T20:59:05.635381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3141703317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3141703317},{"type":"v1","addr":"172.21.15.98:6835","nonce":3141703317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T20:59:27.963 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2024-09-10T20:59:27.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.960+0000 7f590fdec640 1 -- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f58f4077f70 msgr2=0x7f58f407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.960+0000 7f590fdec640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f58f4077f70 0x7f58f407a430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f59000045e0 tx=0x7f590000b260 comp rx=0 tx=0).stop 2024-09-10T20:59:27.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.960+0000 7f590fdec640 1 -- 172.21.15.98:0/1937608446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f590806f330 msgr2=0x7f590806f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:27.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.960+0000 7f590fdec640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f590806f330 0x7f590806f7e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f58f802b470 tx=0x7f58f8004bb0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.960+0000 7f590fdec640 1 -- 172.21.15.98:0/1937608446 shutdown_connections 2024-09-10T20:59:27.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.960+0000 7f590fdec640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f58f4077f70 0x7f58f407a430 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.960+0000 7f590fdec640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f590806f330 0x7f590806f7e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.960+0000 7f590fdec640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59081406c0 0x7f5908070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.960+0000 7f590fdec640 1 --2- 172.21.15.98:0/1937608446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58f0004d40 0x7f590815a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:27.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.960+0000 7f590fdec640 1 -- 172.21.15.98:0/1937608446 >> 172.21.15.98:0/1937608446 conn(0x7f59080b1760 msgr2=0x7f59080ae6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:27.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.960+0000 7f590fdec640 1 -- 172.21.15.98:0/1937608446 shutdown_connections 2024-09-10T20:59:27.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:27.960+0000 7f590fdec640 1 -- 172.21.15.98:0/1937608446 wait complete. 2024-09-10T20:59:27.975 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:59:28.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.073+0000 7f15c9e08640 1 -- 172.21.15.98:0/3100126421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15b0003a00 msgr2=0x7f15b0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.073+0000 7f15c9e08640 1 --2- 172.21.15.98:0/3100126421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15b0003a00 0x7f15b0003e20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f15b400c290 tx=0x7f15b0005830 comp rx=0 tx=0).stop 2024-09-10T20:59:28.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.073+0000 7f15c3fff640 1 -- 172.21.15.98:0/3100126421 reap_dead start 2024-09-10T20:59:28.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.073+0000 7f15c9e08640 1 -- 172.21.15.98:0/3100126421 shutdown_connections 2024-09-10T20:59:28.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.073+0000 7f15c9e08640 1 -- 172.21.15.98:0/3100126421 >> 172.21.15.98:0/3100126421 conn(0x7f15c40d3f70 msgr2=0x7f15c40d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:28.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.073+0000 7f15c9e08640 1 -- 172.21.15.98:0/3100126421 shutdown_connections 2024-09-10T20:59:28.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.073+0000 7f15c9e08640 1 -- 172.21.15.98:0/3100126421 wait complete. 2024-09-10T20:59:28.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.074+0000 7f15c9e08640 1 Processor -- start 2024-09-10T20:59:28.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.074+0000 7f15c9e08640 1 -- start start 2024-09-10T20:59:28.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15c9e08640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b0003a00 0x7f15c41f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:28.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15c9e08640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15c40cb040 0x7f15c41f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:28.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15c9e08640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15c40cb4c0 0x7f15c41f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:28.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15c9e08640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f15c40cea40 con 0x7f15c40cb040 2024-09-10T20:59:28.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15c9e08640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f15c40d2f70 con 0x7f15c40cb4c0 2024-09-10T20:59:28.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15c9e08640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f15c40cebc0 con 0x7f15b0003a00 2024-09-10T20:59:28.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15baffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15c40cb040 0x7f15c41f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:28.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15baffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15c40cb040 0x7f15c41f7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39550/0 (socket says 172.21.15.98:39550) 2024-09-10T20:59:28.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15baffd640 1 -- 172.21.15.98:0/2352257105 learned_addr learned my addr 172.21.15.98:0/2352257105 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:28.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15c3fff640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15c40cb4c0 0x7f15c41f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:28.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15c37fe640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b0003a00 0x7f15c41f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:28.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15baffd640 1 -- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b0003a00 msgr2=0x7f15c41f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15baffd640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b0003a00 0x7f15c41f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15baffd640 1 -- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15c40cb4c0 msgr2=0x7f15c41f80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15baffd640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15c40cb4c0 0x7f15c41f80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15baffd640 1 -- 172.21.15.98:0/2352257105 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c41f89e0 con 0x7f15c40cb040 2024-09-10T20:59:28.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15c37fe640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b0003a00 0x7f15c41f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:28.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.075+0000 7f15baffd640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15c40cb040 0x7f15c41f7bb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f15ac011a50 tx=0x7f15ac011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:28.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.076+0000 7f15c0ff9640 1 -- 172.21.15.98:0/2352257105 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac01f070 con 0x7f15c40cb040 2024-09-10T20:59:28.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.076+0000 7f15c0ff9640 1 -- 172.21.15.98:0/2352257105 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15ac00fdc0 con 0x7f15c40cb040 2024-09-10T20:59:28.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.076+0000 7f15c9e08640 1 -- 172.21.15.98:0/2352257105 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c40cd9e0 con 0x7f15c40cb040 2024-09-10T20:59:28.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.076+0000 7f15c0ff9640 1 -- 172.21.15.98:0/2352257105 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac0126b0 con 0x7f15c40cb040 2024-09-10T20:59:28.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.076+0000 7f15c9e08640 1 -- 172.21.15.98:0/2352257105 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c40ce280 con 0x7f15c40cb040 2024-09-10T20:59:28.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.077+0000 7f15c9e08640 1 -- 172.21.15.98:0/2352257105 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c40d24e0 con 0x7f15c40cb040 2024-09-10T20:59:28.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.077+0000 7f15c0ff9640 1 -- 172.21.15.98:0/2352257105 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f15ac012850 con 0x7f15c40cb040 2024-09-10T20:59:28.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.078+0000 7f15c0ff9640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f15a4077fa0 0x7f15a407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:28.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.078+0000 7f15c0ff9640 1 -- 172.21.15.98:0/2352257105 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6714+0+0 (secure 0 0 0) 0x7f15ac09ad10 con 0x7f15c40cb040 2024-09-10T20:59:28.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.078+0000 7f15c37fe640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f15a4077fa0 0x7f15a407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:28.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.079+0000 7f15c37fe640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f15a4077fa0 0x7f15a407a460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f15b4033000 tx=0x7f15b402d000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:28.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.082+0000 7f15c0ff9640 1 -- 172.21.15.98:0/2352257105 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f15ac062dd0 con 0x7f15c40cb040 2024-09-10T20:59:28.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.256+0000 7f15c9e08640 1 -- 172.21.15.98:0/2352257105 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15c408dfa0 con 0x7f15c40cb040 2024-09-10T20:59:28.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.257+0000 7f15c0ff9640 1 -- 172.21.15.98:0/2352257105 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18969 (secure 0 0 0) 0x7f15ac067800 con 0x7f15c40cb040 2024-09-10T20:59:28.261 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:28.262 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":26,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:59:27.372139+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:58:44.390290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T20:58:45.564099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T20:58:49.250508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:59:28.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.259+0000 7f15c9e08640 1 -- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f15a4077fa0 msgr2=0x7f15a407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.259+0000 7f15c9e08640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f15a4077fa0 0x7f15a407a460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f15b4033000 tx=0x7f15b402d000 comp rx=0 tx=0).stop 2024-09-10T20:59:28.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.259+0000 7f15c9e08640 1 -- 172.21.15.98:0/2352257105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15c40cb040 msgr2=0x7f15c41f7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.259+0000 7f15c9e08640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15c40cb040 0x7f15c41f7bb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f15ac011a50 tx=0x7f15ac011f10 comp rx=0 tx=0).stop 2024-09-10T20:59:28.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.259+0000 7f15c9e08640 1 -- 172.21.15.98:0/2352257105 shutdown_connections 2024-09-10T20:59:28.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.259+0000 7f15c9e08640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f15a4077fa0 0x7f15a407a460 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.259+0000 7f15c9e08640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15c40cb4c0 0x7f15c41f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.259+0000 7f15c9e08640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15c40cb040 0x7f15c41f7bb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.259+0000 7f15c9e08640 1 --2- 172.21.15.98:0/2352257105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15b0003a00 0x7f15c41f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.259+0000 7f15c9e08640 1 -- 172.21.15.98:0/2352257105 >> 172.21.15.98:0/2352257105 conn(0x7f15c40d3f70 msgr2=0x7f15c40b77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:28.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.259+0000 7f15c9e08640 1 -- 172.21.15.98:0/2352257105 shutdown_connections 2024-09-10T20:59:28.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.260+0000 7f15c9e08640 1 -- 172.21.15.98:0/2352257105 wait complete. 2024-09-10T20:59:28.274 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T20:59:28.274 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:59:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.370+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/4201401884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bb80cf210 msgr2=0x7f5bb8072aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.370+0000 7f5bbcb6b640 1 --2- 172.21.15.98:0/4201401884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bb80cf210 0x7f5bb8072aa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009830 tx=0x7f5bb002a480 comp rx=0 tx=0).stop 2024-09-10T20:59:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.370+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/4201401884 shutdown_connections 2024-09-10T20:59:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.370+0000 7f5bbcb6b640 1 --2- 172.21.15.98:0/4201401884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bb80cf210 0x7f5bb8072aa0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.370+0000 7f5bbcb6b640 1 --2- 172.21.15.98:0/4201401884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bb8144780 0x7f5bb80b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.370+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/4201401884 >> 172.21.15.98:0/4201401884 conn(0x7f5bb80b1760 msgr2=0x7f5bb80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.370+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/4201401884 shutdown_connections 2024-09-10T20:59:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.370+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/4201401884 wait complete. 2024-09-10T20:59:28.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.372+0000 7f5bbcb6b640 1 Processor -- start 2024-09-10T20:59:28.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.372+0000 7f5bbcb6b640 1 -- start start 2024-09-10T20:59:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.372+0000 7f5bbcb6b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bb8072680 0x7f5bb81f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.372+0000 7f5bbcb6b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bb80cf210 0x7f5bb80aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.372+0000 7f5bbcb6b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bb8144780 0x7f5bb81e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.372+0000 7f5bbcb6b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb815c9c0 con 0x7f5bb8072680 2024-09-10T20:59:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.372+0000 7f5bbcb6b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb815c840 con 0x7f5bb80cf210 2024-09-10T20:59:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.372+0000 7f5bbcb6b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb815cb40 con 0x7f5bb8144780 2024-09-10T20:59:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5bb659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bb8072680 0x7f5bb81f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5bb659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bb8072680 0x7f5bb81f56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39566/0 (socket says 172.21.15.98:39566) 2024-09-10T20:59:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5bb659c640 1 -- 172.21.15.98:0/2498330408 learned_addr learned my addr 172.21.15.98:0/2498330408 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5bb5d9b640 1 --2- 172.21.15.98:0/2498330408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bb80cf210 0x7f5bb80aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5bb6d9d640 1 --2- 172.21.15.98:0/2498330408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bb8144780 0x7f5bb81e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5bb659c640 1 -- 172.21.15.98:0/2498330408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bb8144780 msgr2=0x7f5bb81e7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5bb659c640 1 --2- 172.21.15.98:0/2498330408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bb8144780 0x7f5bb81e7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5bb659c640 1 -- 172.21.15.98:0/2498330408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bb80cf210 msgr2=0x7f5bb80aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5bb659c640 1 --2- 172.21.15.98:0/2498330408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bb80cf210 0x7f5bb80aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5bb659c640 1 -- 172.21.15.98:0/2498330408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb80af4e0 con 0x7f5bb8072680 2024-09-10T20:59:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5bb659c640 1 --2- 172.21.15.98:0/2498330408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bb8072680 0x7f5bb81f56f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800eb00 tx=0x7f5ba800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5baeffd640 1 -- 172.21.15.98:0/2498330408 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8004b20 con 0x7f5bb8072680 2024-09-10T20:59:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5baeffd640 1 -- 172.21.15.98:0/2498330408 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ba8004ce0 con 0x7f5bb8072680 2024-09-10T20:59:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/2498330408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb80af7c0 con 0x7f5bb8072680 2024-09-10T20:59:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5baeffd640 1 -- 172.21.15.98:0/2498330408 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8005280 con 0x7f5bb8072680 2024-09-10T20:59:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.373+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/2498330408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb80afd80 con 0x7f5bb8072680 2024-09-10T20:59:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.374+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/2498330408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb80b3da0 con 0x7f5bb8072680 2024-09-10T20:59:28.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.375+0000 7f5baeffd640 1 -- 172.21.15.98:0/2498330408 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5ba8005420 con 0x7f5bb8072680 2024-09-10T20:59:28.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.376+0000 7f5baeffd640 1 --2- 172.21.15.98:0/2498330408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5b98077f70 0x7f5b9807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:28.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.376+0000 7f5baeffd640 1 -- 172.21.15.98:0/2498330408 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6714+0+0 (secure 0 0 0) 0x7f5ba809ad80 con 0x7f5bb8072680 2024-09-10T20:59:28.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.376+0000 7f5bb5d9b640 1 --2- 172.21.15.98:0/2498330408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5b98077f70 0x7f5b9807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:28.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.376+0000 7f5bb5d9b640 1 --2- 172.21.15.98:0/2498330408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5b98077f70 0x7f5b9807a430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8141c60 tx=0x7f5ba0007e90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:28.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.380+0000 7f5baeffd640 1 -- 172.21.15.98:0/2498330408 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5ba8062e40 con 0x7f5bb8072680 2024-09-10T20:59:28.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.596+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/2498330408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bb80b4d40 con 0x7f5bb8072680 2024-09-10T20:59:28.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.596+0000 7f5baeffd640 1 -- 172.21.15.98:0/2498330408 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7f5ba8067870 con 0x7f5bb8072680 2024-09-10T20:59:28.601 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:28.601 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"btime":"2024-09-10T20:59:06:294252+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4277,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/1954552281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1954552281},{"type":"v1","addr":"172.21.15.190:6836","nonce":1954552281}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/8997965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":8997965},{"type":"v1","addr":"172.21.15.190:6837","nonce":8997965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1984550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1984550306},{"type":"v1","addr":"172.21.15.98:6833","nonce":1984550306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:58:46.560479+0000","modified":"2024-09-10T20:59:05.635381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3141703317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3141703317},{"type":"v1","addr":"172.21.15.98:6835","nonce":3141703317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T20:59:28.601 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2024-09-10T20:59:28.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.599+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/2498330408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5b98077f70 msgr2=0x7f5b9807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.599+0000 7f5bbcb6b640 1 --2- 172.21.15.98:0/2498330408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5b98077f70 0x7f5b9807a430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8141c60 tx=0x7f5ba0007e90 comp rx=0 tx=0).stop 2024-09-10T20:59:28.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.599+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/2498330408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bb8072680 msgr2=0x7f5bb81f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.599+0000 7f5bbcb6b640 1 --2- 172.21.15.98:0/2498330408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bb8072680 0x7f5bb81f56f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800eb00 tx=0x7f5ba800c2d0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.599+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/2498330408 shutdown_connections 2024-09-10T20:59:28.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.599+0000 7f5bbcb6b640 1 --2- 172.21.15.98:0/2498330408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5b98077f70 0x7f5b9807a430 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.599+0000 7f5bbcb6b640 1 --2- 172.21.15.98:0/2498330408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bb8144780 0x7f5bb81e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.599+0000 7f5bbcb6b640 1 --2- 172.21.15.98:0/2498330408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5bb80cf210 0x7f5bb80aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.599+0000 7f5bbcb6b640 1 --2- 172.21.15.98:0/2498330408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bb8072680 0x7f5bb81f56f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.599+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/2498330408 >> 172.21.15.98:0/2498330408 conn(0x7f5bb80b1760 msgr2=0x7f5bb815f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:28.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.599+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/2498330408 shutdown_connections 2024-09-10T20:59:28.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.599+0000 7f5bbcb6b640 1 -- 172.21.15.98:0/2498330408 wait complete. 2024-09-10T20:59:28.614 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:59:28.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.711+0000 7fcff65a0640 1 -- 172.21.15.98:0/2555170247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff006f140 msgr2=0x7fcff006f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.711+0000 7fcff65a0640 1 --2- 172.21.15.98:0/2555170247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff006f140 0x7fcff006f5f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8005fd0 tx=0x7fcfe802a8a0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.711+0000 7fcff65a0640 1 -- 172.21.15.98:0/2555170247 shutdown_connections 2024-09-10T20:59:28.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.711+0000 7fcff65a0640 1 --2- 172.21.15.98:0/2555170247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff006f140 0x7fcff006f5f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.711+0000 7fcff65a0640 1 --2- 172.21.15.98:0/2555170247 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcff0073780 0x7fcff0073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.711+0000 7fcff65a0640 1 -- 172.21.15.98:0/2555170247 >> 172.21.15.98:0/2555170247 conn(0x7fcff00b1760 msgr2=0x7fcff00b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:28.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.711+0000 7fcff65a0640 1 -- 172.21.15.98:0/2555170247 shutdown_connections 2024-09-10T20:59:28.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.711+0000 7fcff65a0640 1 -- 172.21.15.98:0/2555170247 wait complete. 2024-09-10T20:59:28.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.712+0000 7fcff65a0640 1 Processor -- start 2024-09-10T20:59:28.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.712+0000 7fcff65a0640 1 -- start start 2024-09-10T20:59:28.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.713+0000 7fcff65a0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff006f140 0x7fcff01f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:28.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.713+0000 7fcff65a0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff0073780 0x7fcff015b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:28.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.713+0000 7fcff65a0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcff015c470 0x7fcff00ce430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:28.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.713+0000 7fcff65a0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcff006fcb0 con 0x7fcff0073780 2024-09-10T20:59:28.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.713+0000 7fcff65a0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcff006fb30 con 0x7fcff006f140 2024-09-10T20:59:28.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.713+0000 7fcff65a0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcff006fe30 con 0x7fcff015c470 2024-09-10T20:59:28.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.713+0000 7fcfef7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff0073780 0x7fcff015b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:28.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.713+0000 7fcfef7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff0073780 0x7fcff015b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39590/0 (socket says 172.21.15.98:39590) 2024-09-10T20:59:28.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.713+0000 7fcfef7fe640 1 -- 172.21.15.98:0/2655120843 learned_addr learned my addr 172.21.15.98:0/2655120843 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:28.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.713+0000 7fcff4b3d640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcff015c470 0x7fcff00ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.713+0000 7fcfeffff640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff006f140 0x7fcff01f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.714+0000 7fcfef7fe640 1 -- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcff015c470 msgr2=0x7fcff00ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.714+0000 7fcfef7fe640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcff015c470 0x7fcff00ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.714+0000 7fcfef7fe640 1 -- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff006f140 msgr2=0x7fcff01f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.714+0000 7fcfef7fe640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff006f140 0x7fcff01f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.714+0000 7fcfef7fe640 1 -- 172.21.15.98:0/2655120843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff0130d50 con 0x7fcff0073780 2024-09-10T20:59:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.714+0000 7fcfeffff640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff006f140 0x7fcff01f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.714+0000 7fcfef7fe640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff0073780 0x7fcff015b2a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400c340 tx=0x7fcfe400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.714+0000 7fcfecff9640 1 -- 172.21.15.98:0/2655120843 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4017070 con 0x7fcff0073780 2024-09-10T20:59:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.714+0000 7fcfecff9640 1 -- 172.21.15.98:0/2655120843 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfe40035f0 con 0x7fcff0073780 2024-09-10T20:59:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.714+0000 7fcff65a0640 1 -- 172.21.15.98:0/2655120843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff0131030 con 0x7fcff0073780 2024-09-10T20:59:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.714+0000 7fcfecff9640 1 -- 172.21.15.98:0/2655120843 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4003a80 con 0x7fcff0073780 2024-09-10T20:59:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.714+0000 7fcff65a0640 1 -- 172.21.15.98:0/2655120843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff01315f0 con 0x7fcff0073780 2024-09-10T20:59:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.715+0000 7fcff65a0640 1 -- 172.21.15.98:0/2655120843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff006f5f0 con 0x7fcff0073780 2024-09-10T20:59:28.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.716+0000 7fcfecff9640 1 -- 172.21.15.98:0/2655120843 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcfe4003c20 con 0x7fcff0073780 2024-09-10T20:59:28.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.716+0000 7fcfecff9640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcfdc077f70 0x7fcfdc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:28.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.716+0000 7fcfecff9640 1 -- 172.21.15.98:0/2655120843 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fcfe40996b0 con 0x7fcff0073780 2024-09-10T20:59:28.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.716+0000 7fcfeffff640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcfdc077f70 0x7fcfdc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:28.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.717+0000 7fcfeffff640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcfdc077f70 0x7fcfdc07a430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000baa0 tx=0x7fcfe0005d90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:28.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.720+0000 7fcfecff9640 1 -- 172.21.15.98:0/2655120843 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fcfe4061dc0 con 0x7fcff0073780 2024-09-10T20:59:28.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.935+0000 7fcff65a0640 1 -- 172.21.15.98:0/2655120843 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcff012ed40 con 0x7fcff0073780 2024-09-10T20:59:28.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.935+0000 7fcfecff9640 1 -- 172.21.15.98:0/2655120843 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5216 (secure 0 0 0) 0x7fcfe40616c0 con 0x7fcff0073780 2024-09-10T20:59:28.940 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:28.940 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":8,"btime":"2024-09-10T20:59:06:294252+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4277,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6836/1954552281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1954552281},{"type":"v1","addr":"172.21.15.190:6836","nonce":1954552281}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4348,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/8997965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6835","nonce":8997965},{"type":"v1","addr":"172.21.15.190:6837","nonce":8997965}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1984550306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1984550306},{"type":"v1","addr":"172.21.15.98:6833","nonce":1984550306}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:58:46.560479+0000","modified":"2024-09-10T20:59:05.635381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4380},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4380":{"gid":4380,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3141703317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3141703317},{"type":"v1","addr":"172.21.15.98:6835","nonce":3141703317}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4380,"qdb_cluster":[4380]},"id":1}]} 2024-09-10T20:59:28.940 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 8 2024-09-10T20:59:28.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.938+0000 7fcff65a0640 1 -- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcfdc077f70 msgr2=0x7fcfdc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.938+0000 7fcff65a0640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcfdc077f70 0x7fcfdc07a430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000baa0 tx=0x7fcfe0005d90 comp rx=0 tx=0).stop 2024-09-10T20:59:28.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.938+0000 7fcff65a0640 1 -- 172.21.15.98:0/2655120843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff0073780 msgr2=0x7fcff015b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:28.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.938+0000 7fcff65a0640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff0073780 0x7fcff015b2a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400c340 tx=0x7fcfe400c800 comp rx=0 tx=0).stop 2024-09-10T20:59:28.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.938+0000 7fcff65a0640 1 -- 172.21.15.98:0/2655120843 shutdown_connections 2024-09-10T20:59:28.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.938+0000 7fcff65a0640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcfdc077f70 0x7fcfdc07a430 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.938+0000 7fcff65a0640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcff015c470 0x7fcff00ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.938+0000 7fcff65a0640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff0073780 0x7fcff015b2a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.938+0000 7fcff65a0640 1 --2- 172.21.15.98:0/2655120843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff006f140 0x7fcff01f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:28.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.938+0000 7fcff65a0640 1 -- 172.21.15.98:0/2655120843 >> 172.21.15.98:0/2655120843 conn(0x7fcff00b1760 msgr2=0x7fcff015c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:28.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.938+0000 7fcff65a0640 1 -- 172.21.15.98:0/2655120843 shutdown_connections 2024-09-10T20:59:28.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:28.938+0000 7fcff65a0640 1 -- 172.21.15.98:0/2655120843 wait complete. 2024-09-10T20:59:28.953 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:59:29.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.049+0000 7f85382b6640 1 -- 172.21.15.98:0/648370520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f851c003a50 msgr2=0x7f851c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.049+0000 7f85382b6640 1 --2- 172.21.15.98:0/648370520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f851c003a50 0x7f851c003e70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f852c0016c0 tx=0x7f851c005830 comp rx=0 tx=0).stop 2024-09-10T20:59:29.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.049+0000 7f85382b6640 1 -- 172.21.15.98:0/648370520 shutdown_connections 2024-09-10T20:59:29.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.049+0000 7f85382b6640 1 --2- 172.21.15.98:0/648370520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f851c003a50 0x7f851c003e70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.049+0000 7f85382b6640 1 -- 172.21.15.98:0/648370520 >> 172.21.15.98:0/648370520 conn(0x7f8530074150 msgr2=0x7f8530074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:29.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.049+0000 7f85382b6640 1 -- 172.21.15.98:0/648370520 shutdown_connections 2024-09-10T20:59:29.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.049+0000 7f85382b6640 1 -- 172.21.15.98:0/648370520 wait complete. 2024-09-10T20:59:29.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.050+0000 7f85382b6640 1 Processor -- start 2024-09-10T20:59:29.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.050+0000 7f85382b6640 1 -- start start 2024-09-10T20:59:29.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.051+0000 7f85382b6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f851c003a50 0x7f8530155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.051+0000 7f85382b6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8530159490 0x7f85301f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.051+0000 7f85382b6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8530158c80 0x7f853015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.051+0000 7f85382b6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f85300b5ad0 con 0x7f851c003a50 2024-09-10T20:59:29.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.051+0000 7f85382b6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f85300b5950 con 0x7f8530158c80 2024-09-10T20:59:29.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.051+0000 7f85382b6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f85300b5c50 con 0x7f8530159490 2024-09-10T20:59:29.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.051+0000 7f8536052640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f851c003a50 0x7f8530155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.051+0000 7f8536052640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f851c003a50 0x7f8530155f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39602/0 (socket says 172.21.15.98:39602) 2024-09-10T20:59:29.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.051+0000 7f8536052640 1 -- 172.21.15.98:0/3005500591 learned_addr learned my addr 172.21.15.98:0/3005500591 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:29.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.051+0000 7f8536853640 1 --2- 172.21.15.98:0/3005500591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8530158c80 0x7f853015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.051+0000 7f8535851640 1 --2- 172.21.15.98:0/3005500591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8530159490 0x7f85301f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.051+0000 7f8536052640 1 -- 172.21.15.98:0/3005500591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8530159490 msgr2=0x7f85301f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.051+0000 7f8536052640 1 --2- 172.21.15.98:0/3005500591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8530159490 0x7f85301f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.052+0000 7f8536052640 1 -- 172.21.15.98:0/3005500591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8530158c80 msgr2=0x7f853015bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.052+0000 7f8536052640 1 --2- 172.21.15.98:0/3005500591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8530158c80 0x7f853015bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.052+0000 7f8536052640 1 -- 172.21.15.98:0/3005500591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85300ce0d0 con 0x7f851c003a50 2024-09-10T20:59:29.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.052+0000 7f8536052640 1 --2- 172.21.15.98:0/3005500591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f851c003a50 0x7f8530155f10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f852c02a910 tx=0x7f852c0063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:29.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.052+0000 7f8526ffd640 1 -- 172.21.15.98:0/3005500591 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c003a00 con 0x7f851c003a50 2024-09-10T20:59:29.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.052+0000 7f8526ffd640 1 -- 172.21.15.98:0/3005500591 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f852c039070 con 0x7f851c003a50 2024-09-10T20:59:29.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.052+0000 7f85382b6640 1 -- 172.21.15.98:0/3005500591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85300ce3b0 con 0x7f851c003a50 2024-09-10T20:59:29.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.052+0000 7f8526ffd640 1 -- 172.21.15.98:0/3005500591 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c033420 con 0x7f851c003a50 2024-09-10T20:59:29.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.052+0000 7f85382b6640 1 -- 172.21.15.98:0/3005500591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85300ce970 con 0x7f851c003a50 2024-09-10T20:59:29.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.053+0000 7f85382b6640 1 -- 172.21.15.98:0/3005500591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8530069a50 con 0x7f851c003a50 2024-09-10T20:59:29.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.053+0000 7f8526ffd640 1 -- 172.21.15.98:0/3005500591 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f852c0335c0 con 0x7f851c003a50 2024-09-10T20:59:29.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.054+0000 7f8526ffd640 1 --2- 172.21.15.98:0/3005500591 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8514077fa0 0x7f851407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.054+0000 7f8526ffd640 1 -- 172.21.15.98:0/3005500591 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f852c0b9aa0 con 0x7f851c003a50 2024-09-10T20:59:29.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.054+0000 7f8535851640 1 --2- 172.21.15.98:0/3005500591 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8514077fa0 0x7f851407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.055+0000 7f8535851640 1 --2- 172.21.15.98:0/3005500591 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8514077fa0 0x7f851407a460 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85301f8450 tx=0x7f8520005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:29.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.058+0000 7f8526ffd640 1 -- 172.21.15.98:0/3005500591 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f852c0821b0 con 0x7f851c003a50 2024-09-10T20:59:29.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.233+0000 7f85382b6640 1 -- 172.21.15.98:0/3005500591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85300b61b0 con 0x7f851c003a50 2024-09-10T20:59:29.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.234+0000 7f8526ffd640 1 -- 172.21.15.98:0/3005500591 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18969 (secure 0 0 0) 0x7f852c081ab0 con 0x7f851c003a50 2024-09-10T20:59:29.239 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:29.239 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":27,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:59:28.376486+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:58:44.390290+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T20:58:45.564099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T20:58:49.250508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.d","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:59:29.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.236+0000 7f85382b6640 1 -- 172.21.15.98:0/3005500591 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8514077fa0 msgr2=0x7f851407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.236+0000 7f85382b6640 1 --2- 172.21.15.98:0/3005500591 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8514077fa0 0x7f851407a460 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85301f8450 tx=0x7f8520005f90 comp rx=0 tx=0).stop 2024-09-10T20:59:29.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.236+0000 7f85382b6640 1 -- 172.21.15.98:0/3005500591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f851c003a50 msgr2=0x7f8530155f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.236+0000 7f85382b6640 1 --2- 172.21.15.98:0/3005500591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f851c003a50 0x7f8530155f10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f852c02a910 tx=0x7f852c0063a0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.236+0000 7f85382b6640 1 -- 172.21.15.98:0/3005500591 shutdown_connections 2024-09-10T20:59:29.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.236+0000 7f85382b6640 1 --2- 172.21.15.98:0/3005500591 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8514077fa0 0x7f851407a460 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.236+0000 7f85382b6640 1 --2- 172.21.15.98:0/3005500591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8530158c80 0x7f853015bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.236+0000 7f85382b6640 1 --2- 172.21.15.98:0/3005500591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8530159490 0x7f85301f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.236+0000 7f85382b6640 1 --2- 172.21.15.98:0/3005500591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f851c003a50 0x7f8530155f10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.236+0000 7f85382b6640 1 -- 172.21.15.98:0/3005500591 >> 172.21.15.98:0/3005500591 conn(0x7f8530074150 msgr2=0x7f85300af320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:29.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.236+0000 7f85382b6640 1 -- 172.21.15.98:0/3005500591 shutdown_connections 2024-09-10T20:59:29.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.236+0000 7f85382b6640 1 -- 172.21.15.98:0/3005500591 wait complete. 2024-09-10T20:59:29.252 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T20:59:29.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.350+0000 7fd6faacd640 1 -- 172.21.15.98:0/2196545701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6e4004d40 msgr2=0x7fd6e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.350+0000 7fd6faacd640 1 --2- 172.21.15.98:0/2196545701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6e4004d40 0x7fd6e40051c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009e20 tx=0x7fd6e4005830 comp rx=0 tx=0).stop 2024-09-10T20:59:29.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.350+0000 7fd6f906a640 1 -- 172.21.15.98:0/2196545701 reap_dead start 2024-09-10T20:59:29.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.350+0000 7fd6faacd640 1 -- 172.21.15.98:0/2196545701 shutdown_connections 2024-09-10T20:59:29.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.350+0000 7fd6faacd640 1 -- 172.21.15.98:0/2196545701 >> 172.21.15.98:0/2196545701 conn(0x7fd6f40b1760 msgr2=0x7fd6f40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:29.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.350+0000 7fd6faacd640 1 -- 172.21.15.98:0/2196545701 shutdown_connections 2024-09-10T20:59:29.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.350+0000 7fd6faacd640 1 -- 172.21.15.98:0/2196545701 wait complete. 2024-09-10T20:59:29.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.351+0000 7fd6faacd640 1 Processor -- start 2024-09-10T20:59:29.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.351+0000 7fd6faacd640 1 -- start start 2024-09-10T20:59:29.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.352+0000 7fd6faacd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6e4004d40 0x7fd6f413e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.352+0000 7fd6faacd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f413d9d0 0x7fd6f41e6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.352+0000 7fd6faacd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f415aca0 0x7fd6f415b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.352+0000 7fd6faacd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f4073ce0 con 0x7fd6e4004d40 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.352+0000 7fd6faacd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f4073b60 con 0x7fd6f413d9d0 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.352+0000 7fd6faacd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f4073e60 con 0x7fd6f415aca0 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.352+0000 7fd6f8869640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6e4004d40 0x7fd6f413e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.352+0000 7fd6f8869640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6e4004d40 0x7fd6f413e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39608/0 (socket says 172.21.15.98:39608) 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.352+0000 7fd6f8869640 1 -- 172.21.15.98:0/1211539196 learned_addr learned my addr 172.21.15.98:0/1211539196 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.353+0000 7fd6f3fff640 1 --2- 172.21.15.98:0/1211539196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f413d9d0 0x7fd6f41e6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.353+0000 7fd6f906a640 1 --2- 172.21.15.98:0/1211539196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f415aca0 0x7fd6f415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.353+0000 7fd6f8869640 1 -- 172.21.15.98:0/1211539196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f415aca0 msgr2=0x7fd6f415b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.353+0000 7fd6f8869640 1 --2- 172.21.15.98:0/1211539196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f415aca0 0x7fd6f415b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.353+0000 7fd6f8869640 1 -- 172.21.15.98:0/1211539196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f413d9d0 msgr2=0x7fd6f41e6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.353+0000 7fd6f8869640 1 --2- 172.21.15.98:0/1211539196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f413d9d0 0x7fd6f41e6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.353+0000 7fd6f8869640 1 -- 172.21.15.98:0/1211539196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f41e5db0 con 0x7fd6e4004d40 2024-09-10T20:59:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.353+0000 7fd6f8869640 1 --2- 172.21.15.98:0/1211539196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6e4004d40 0x7fd6f413e6e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8004650 tx=0x7fd6e800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.353+0000 7fd6f17fa640 1 -- 172.21.15.98:0/1211539196 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8010850 con 0x7fd6e4004d40 2024-09-10T20:59:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.353+0000 7fd6f17fa640 1 -- 172.21.15.98:0/1211539196 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6e80109f0 con 0x7fd6e4004d40 2024-09-10T20:59:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.353+0000 7fd6f17fa640 1 -- 172.21.15.98:0/1211539196 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8010e20 con 0x7fd6e4004d40 2024-09-10T20:59:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.353+0000 7fd6faacd640 1 -- 172.21.15.98:0/1211539196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f41e6090 con 0x7fd6e4004d40 2024-09-10T20:59:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.353+0000 7fd6faacd640 1 -- 172.21.15.98:0/1211539196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f41e6650 con 0x7fd6e4004d40 2024-09-10T20:59:29.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.354+0000 7fd6faacd640 1 -- 172.21.15.98:0/1211539196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f406f5f0 con 0x7fd6e4004d40 2024-09-10T20:59:29.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.355+0000 7fd6f17fa640 1 -- 172.21.15.98:0/1211539196 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd6e8023440 con 0x7fd6e4004d40 2024-09-10T20:59:29.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.355+0000 7fd6f17fa640 1 --2- 172.21.15.98:0/1211539196 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd6d8077f70 0x7fd6d807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.355+0000 7fd6f17fa640 1 -- 172.21.15.98:0/1211539196 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fd6e809ac70 con 0x7fd6e4004d40 2024-09-10T20:59:29.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.355+0000 7fd6f3fff640 1 --2- 172.21.15.98:0/1211539196 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd6d8077f70 0x7fd6d807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.356+0000 7fd6f3fff640 1 --2- 172.21.15.98:0/1211539196 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd6d8077f70 0x7fd6d807a430 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0001d50 tx=0x7fd6e0000fd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:29.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.359+0000 7fd6f17fa640 1 -- 172.21.15.98:0/1211539196 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd6e8062d40 con 0x7fd6e4004d40 2024-09-10T20:59:29.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.571+0000 7fd6faacd640 1 -- 172.21.15.98:0/1211539196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd6f40b1da0 con 0x7fd6e4004d40 2024-09-10T20:59:29.575 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-10T20:59:29.571+0000 7fb4184a9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T20:59:29.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.573+0000 7fd6f17fa640 1 -- 172.21.15.98:0/1211539196 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7fd6e8067770 con 0x7fd6e4004d40 2024-09-10T20:59:29.577 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T20:59:29.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.575+0000 7fd6faacd640 1 -- 172.21.15.98:0/1211539196 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd6d8077f70 msgr2=0x7fd6d807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.575+0000 7fd6faacd640 1 --2- 172.21.15.98:0/1211539196 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd6d8077f70 0x7fd6d807a430 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0001d50 tx=0x7fd6e0000fd0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.575+0000 7fd6faacd640 1 -- 172.21.15.98:0/1211539196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6e4004d40 msgr2=0x7fd6f413e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.575+0000 7fd6faacd640 1 --2- 172.21.15.98:0/1211539196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6e4004d40 0x7fd6f413e6e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8004650 tx=0x7fd6e800ba40 comp rx=0 tx=0).stop 2024-09-10T20:59:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.575+0000 7fd6faacd640 1 -- 172.21.15.98:0/1211539196 shutdown_connections 2024-09-10T20:59:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.575+0000 7fd6faacd640 1 --2- 172.21.15.98:0/1211539196 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd6d8077f70 0x7fd6d807a430 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.575+0000 7fd6faacd640 1 --2- 172.21.15.98:0/1211539196 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6f415aca0 0x7fd6f415b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.575+0000 7fd6faacd640 1 --2- 172.21.15.98:0/1211539196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6f413d9d0 0x7fd6f41e6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.575+0000 7fd6faacd640 1 --2- 172.21.15.98:0/1211539196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6e4004d40 0x7fd6f413e6e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.575+0000 7fd6faacd640 1 -- 172.21.15.98:0/1211539196 >> 172.21.15.98:0/1211539196 conn(0x7fd6f40b1760 msgr2=0x7fd6f40cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.575+0000 7fd6faacd640 1 -- 172.21.15.98:0/1211539196 shutdown_connections 2024-09-10T20:59:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.576+0000 7fd6faacd640 1 -- 172.21.15.98:0/1211539196 wait complete. 2024-09-10T20:59:29.581 INFO:tasks.ceph.mds.c.smithi098.stderr: -4385> 2024-09-10T20:58:52.197+0000 7fc5824a7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T20:59:29.591 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T20:59:29.612 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-10T20:59:29.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.691+0000 7f3892910640 1 -- 172.21.15.98:0/2923009482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388c13cf80 msgr2=0x7f388c06f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.691+0000 7f3892910640 1 --2- 172.21.15.98:0/2923009482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388c13cf80 0x7f388c06f750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f387c00baf0 tx=0x7f387c02ad30 comp rx=0 tx=0).stop 2024-09-10T20:59:29.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.691+0000 7f3892910640 1 -- 172.21.15.98:0/2923009482 shutdown_connections 2024-09-10T20:59:29.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.691+0000 7f3892910640 1 --2- 172.21.15.98:0/2923009482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15e140 0x7f388c06fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.691+0000 7f3892910640 1 --2- 172.21.15.98:0/2923009482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388c13cf80 0x7f388c06f750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.691+0000 7f3892910640 1 -- 172.21.15.98:0/2923009482 >> 172.21.15.98:0/2923009482 conn(0x7f388c0b1760 msgr2=0x7f388c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:29.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.691+0000 7f3892910640 1 -- 172.21.15.98:0/2923009482 shutdown_connections 2024-09-10T20:59:29.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.691+0000 7f3892910640 1 -- 172.21.15.98:0/2923009482 wait complete. 2024-09-10T20:59:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.692+0000 7f3892910640 1 Processor -- start 2024-09-10T20:59:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.693+0000 7f3892910640 1 -- start start 2024-09-10T20:59:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.693+0000 7f3892910640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f388c13cf80 0x7f388c145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.693+0000 7f3892910640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15b1e0 0x7f388c147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.693+0000 7f3892910640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388c15e140 0x7f388c1f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.693+0000 7f3892910640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f388c0b3e90 con 0x7f388c13cf80 2024-09-10T20:59:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.693+0000 7f3892910640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f388c0b3d10 con 0x7f388c15e140 2024-09-10T20:59:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.693+0000 7f3892910640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f388c0b4010 con 0x7f388c15b1e0 2024-09-10T20:59:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.693+0000 7f388bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f388c13cf80 0x7f388c145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.693+0000 7f388bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f388c13cf80 0x7f388c145b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39638/0 (socket says 172.21.15.98:39638) 2024-09-10T20:59:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.693+0000 7f388bfff640 1 -- 172.21.15.98:0/4063286546 learned_addr learned my addr 172.21.15.98:0/4063286546 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.693+0000 7f3890ead640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388c15e140 0x7f388c1f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.694+0000 7f388b7fe640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15b1e0 0x7f388c147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.694+0000 7f388bfff640 1 -- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15b1e0 msgr2=0x7f388c147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.694+0000 7f388bfff640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15b1e0 0x7f388c147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.694+0000 7f388bfff640 1 -- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388c15e140 msgr2=0x7f388c1f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.694+0000 7f388bfff640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388c15e140 0x7f388c1f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.694+0000 7f388bfff640 1 -- 172.21.15.98:0/4063286546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c13b370 con 0x7f388c13cf80 2024-09-10T20:59:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.694+0000 7f3890ead640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388c15e140 0x7f388c1f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.694+0000 7f388bfff640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f388c13cf80 0x7f388c145b30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f387c00bc10 tx=0x7f387c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.694+0000 7f3888ff9640 1 -- 172.21.15.98:0/4063286546 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c038040 con 0x7f388c13cf80 2024-09-10T20:59:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.694+0000 7f3892910640 1 -- 172.21.15.98:0/4063286546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388c1f65c0 con 0x7f388c13cf80 2024-09-10T20:59:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.694+0000 7f3888ff9640 1 -- 172.21.15.98:0/4063286546 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f387c00b4d0 con 0x7f388c13cf80 2024-09-10T20:59:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.694+0000 7f3888ff9640 1 -- 172.21.15.98:0/4063286546 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c00b920 con 0x7f388c13cf80 2024-09-10T20:59:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.694+0000 7f3892910640 1 -- 172.21.15.98:0/4063286546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388c13b5e0 con 0x7f388c13cf80 2024-09-10T20:59:29.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.695+0000 7f3892910640 1 -- 172.21.15.98:0/4063286546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388c0b32f0 con 0x7f388c13cf80 2024-09-10T20:59:29.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.696+0000 7f3888ff9640 1 -- 172.21.15.98:0/4063286546 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f387c03b070 con 0x7f388c13cf80 2024-09-10T20:59:29.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.696+0000 7f3888ff9640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3878077fa0 0x7f387807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.696+0000 7f3888ff9640 1 -- 172.21.15.98:0/4063286546 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6800+0+0 (secure 0 0 0) 0x7f387c0ba080 con 0x7f388c13cf80 2024-09-10T20:59:29.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.696+0000 7f388b7fe640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3878077fa0 0x7f387807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.697+0000 7f388b7fe640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3878077fa0 0x7f387807a460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f388c15c6a0 tx=0x7f3880009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.700+0000 7f3888ff9640 1 -- 172.21.15.98:0/4063286546 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f388c0b32f0 con 0x7f388c13cf80 2024-09-10T20:59:29.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:29.914+0000 7f3892910640 1 -- 172.21.15.98:0/4063286546 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f388c15da30 con 0x7f388c13cf80 2024-09-10T20:59:30.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.574+0000 7f3888ff9640 1 -- 172.21.15.98:0/4063286546 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f387c089e20 con 0x7f388c13cf80 2024-09-10T20:59:30.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.578+0000 7f3892910640 1 -- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3878077fa0 msgr2=0x7f387807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:30.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.578+0000 7f3892910640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3878077fa0 0x7f387807a460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f388c15c6a0 tx=0x7f3880009040 comp rx=0 tx=0).stop 2024-09-10T20:59:30.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.578+0000 7f3892910640 1 -- 172.21.15.98:0/4063286546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f388c13cf80 msgr2=0x7f388c145b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:30.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.578+0000 7f3892910640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f388c13cf80 0x7f388c145b30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f387c00bc10 tx=0x7f387c0045e0 comp rx=0 tx=0).stop 2024-09-10T20:59:30.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.578+0000 7f3892910640 1 -- 172.21.15.98:0/4063286546 shutdown_connections 2024-09-10T20:59:30.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.578+0000 7f3892910640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3878077fa0 0x7f387807a460 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:30.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.578+0000 7f3892910640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388c15e140 0x7f388c1f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:30.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.578+0000 7f3892910640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f388c15b1e0 0x7f388c147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:30.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.578+0000 7f3892910640 1 --2- 172.21.15.98:0/4063286546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f388c13cf80 0x7f388c145b30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:30.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.578+0000 7f3892910640 1 -- 172.21.15.98:0/4063286546 >> 172.21.15.98:0/4063286546 conn(0x7f388c0b1760 msgr2=0x7f388c0cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:30.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.578+0000 7f3892910640 1 -- 172.21.15.98:0/4063286546 shutdown_connections 2024-09-10T20:59:30.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.578+0000 7f3892910640 1 -- 172.21.15.98:0/4063286546 wait complete. 2024-09-10T20:59:30.597 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T20:59:30.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.743+0000 7fea5fc52640 1 -- 172.21.15.98:0/177488578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea5815b1e0 msgr2=0x7fea5815da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:30.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.743+0000 7fea5fc52640 1 --2- 172.21.15.98:0/177488578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea5815b1e0 0x7fea5815da30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fea58151530 tx=0x7fea4c02b7f0 comp rx=0 tx=0).stop 2024-09-10T20:59:30.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.743+0000 7fea5fc52640 1 -- 172.21.15.98:0/177488578 shutdown_connections 2024-09-10T20:59:30.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.743+0000 7fea5fc52640 1 --2- 172.21.15.98:0/177488578 >> v2:172.21.15.190:3300/0 conn(0x7fea5813cf80 0x7fea580b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:30.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.743+0000 7fea5fc52640 1 --2- 172.21.15.98:0/177488578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea5815b1e0 0x7fea5815da30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:30.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.743+0000 7fea5fc52640 1 -- 172.21.15.98:0/177488578 >> 172.21.15.98:0/177488578 conn(0x7fea580b1760 msgr2=0x7fea580b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:30.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.744+0000 7fea5fc52640 1 -- 172.21.15.98:0/177488578 shutdown_connections 2024-09-10T20:59:30.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.744+0000 7fea5fc52640 1 -- 172.21.15.98:0/177488578 wait complete. 2024-09-10T20:59:30.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.745+0000 7fea5fc52640 1 Processor -- start 2024-09-10T20:59:30.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.745+0000 7fea5fc52640 1 -- start start 2024-09-10T20:59:30.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.745+0000 7fea5fc52640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea58070650 0x7fea581f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:30.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.745+0000 7fea5fc52640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea5813cf80 0x7fea581530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:30.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.745+0000 7fea5fc52640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea5815b1e0 0x7fea58153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:30.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.745+0000 7fea5fc52640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fea58070030 con 0x7fea5815b1e0 2024-09-10T20:59:30.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.745+0000 7fea5fc52640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fea5806feb0 con 0x7fea5813cf80 2024-09-10T20:59:30.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.745+0000 7fea5fc52640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fea580b4250 con 0x7fea58070650 2024-09-10T20:59:30.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea5e1ef640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea5815b1e0 0x7fea58153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:30.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea5e1ef640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea5815b1e0 0x7fea58153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:52870/0 (socket says 172.21.15.98:52870) 2024-09-10T20:59:30.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea5e1ef640 1 -- 172.21.15.98:0/3456536145 learned_addr learned my addr 172.21.15.98:0/3456536145 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:30.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea5d9ee640 1 --2- 172.21.15.98:0/3456536145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea58070650 0x7fea581f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:30.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea5d1ed640 1 --2- 172.21.15.98:0/3456536145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea5813cf80 0x7fea581530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:30.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea5e1ef640 1 -- 172.21.15.98:0/3456536145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea58070650 msgr2=0x7fea581f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:30.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea5e1ef640 1 --2- 172.21.15.98:0/3456536145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea58070650 0x7fea581f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:30.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea5e1ef640 1 -- 172.21.15.98:0/3456536145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea5813cf80 msgr2=0x7fea581530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:30.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea5e1ef640 1 --2- 172.21.15.98:0/3456536145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea5813cf80 0x7fea581530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:30.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea5e1ef640 1 -- 172.21.15.98:0/3456536145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea581f7010 con 0x7fea5815b1e0 2024-09-10T20:59:30.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea5e1ef640 1 --2- 172.21.15.98:0/3456536145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea5815b1e0 0x7fea58153630 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fea5000cad0 tx=0x7fea5000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:30.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea4a7fc640 1 -- 172.21.15.98:0/3456536145 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5000ae20 con 0x7fea5815b1e0 2024-09-10T20:59:30.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea4a7fc640 1 -- 172.21.15.98:0/3456536145 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea50004b20 con 0x7fea5815b1e0 2024-09-10T20:59:30.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea5fc52640 1 -- 172.21.15.98:0/3456536145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea581450e0 con 0x7fea5815b1e0 2024-09-10T20:59:30.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea4a7fc640 1 -- 172.21.15.98:0/3456536145 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50005020 con 0x7fea5815b1e0 2024-09-10T20:59:30.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.746+0000 7fea5fc52640 1 -- 172.21.15.98:0/3456536145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea581456a0 con 0x7fea5815b1e0 2024-09-10T20:59:30.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.747+0000 7fea5fc52640 1 -- 172.21.15.98:0/3456536145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea580b3d10 con 0x7fea5815b1e0 2024-09-10T20:59:30.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.748+0000 7fea4a7fc640 1 -- 172.21.15.98:0/3456536145 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fea500051c0 con 0x7fea5815b1e0 2024-09-10T20:59:30.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.748+0000 7fea4a7fc640 1 --2- 172.21.15.98:0/3456536145 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fea3c077f70 0x7fea3c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:30.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.748+0000 7fea4a7fc640 1 -- 172.21.15.98:0/3456536145 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6800+0+0 (secure 0 0 0) 0x7fea5009a890 con 0x7fea5815b1e0 2024-09-10T20:59:30.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.748+0000 7fea5d9ee640 1 --2- 172.21.15.98:0/3456536145 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fea3c077f70 0x7fea3c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:30.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.749+0000 7fea5d9ee640 1 --2- 172.21.15.98:0/3456536145 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fea3c077f70 0x7fea3c07a430 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea4c035000 tx=0x7fea4c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:30.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.752+0000 7fea4a7fc640 1 -- 172.21.15.98:0/3456536145 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fea581f7010 con 0x7fea5815b1e0 2024-09-10T20:59:30.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:30.927+0000 7fea5fc52640 1 -- 172.21.15.98:0/3456536145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fea58153d30 con 0x7fea5815b1e0 2024-09-10T20:59:31.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.581+0000 7fea4a7fc640 1 -- 172.21.15.98:0/3456536145 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7fea50014030 con 0x7fea5815b1e0 2024-09-10T20:59:31.586 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-09-10T20:59:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.584+0000 7fea5fc52640 1 -- 172.21.15.98:0/3456536145 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fea3c077f70 msgr2=0x7fea3c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.584+0000 7fea5fc52640 1 --2- 172.21.15.98:0/3456536145 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fea3c077f70 0x7fea3c07a430 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea4c035000 tx=0x7fea4c004730 comp rx=0 tx=0).stop 2024-09-10T20:59:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.584+0000 7fea5fc52640 1 -- 172.21.15.98:0/3456536145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea5815b1e0 msgr2=0x7fea58153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.584+0000 7fea5fc52640 1 --2- 172.21.15.98:0/3456536145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea5815b1e0 0x7fea58153630 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fea5000cad0 tx=0x7fea5000cf90 comp rx=0 tx=0).stop 2024-09-10T20:59:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.584+0000 7fea5fc52640 1 -- 172.21.15.98:0/3456536145 shutdown_connections 2024-09-10T20:59:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.584+0000 7fea5fc52640 1 --2- 172.21.15.98:0/3456536145 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fea3c077f70 0x7fea3c07a430 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.584+0000 7fea5fc52640 1 --2- 172.21.15.98:0/3456536145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea5815b1e0 0x7fea58153630 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.584+0000 7fea5fc52640 1 --2- 172.21.15.98:0/3456536145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea5813cf80 0x7fea581530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.584+0000 7fea5fc52640 1 --2- 172.21.15.98:0/3456536145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea58070650 0x7fea581f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.584+0000 7fea5fc52640 1 -- 172.21.15.98:0/3456536145 >> 172.21.15.98:0/3456536145 conn(0x7fea580b1760 msgr2=0x7fea58158670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.584+0000 7fea5fc52640 1 -- 172.21.15.98:0/3456536145 shutdown_connections 2024-09-10T20:59:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.584+0000 7fea5fc52640 1 -- 172.21.15.98:0/3456536145 wait complete. 2024-09-10T20:59:31.601 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T20:59:31.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.705+0000 7f6395260640 1 -- 172.21.15.98:0/3695023160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6390074c70 msgr2=0x7f63901600a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:31.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.705+0000 7f6395260640 1 --2- 172.21.15.98:0/3695023160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6390074c70 0x7f63901600a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6384009830 tx=0x7f638402a510 comp rx=0 tx=0).stop 2024-09-10T20:59:31.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.705+0000 7f6395260640 1 -- 172.21.15.98:0/3695023160 shutdown_connections 2024-09-10T20:59:31.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.705+0000 7f6395260640 1 --2- 172.21.15.98:0/3695023160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6390074c70 0x7f63901600a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:31.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.705+0000 7f6395260640 1 --2- 172.21.15.98:0/3695023160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f639015bdb0 0x7f63900b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:31.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.705+0000 7f6395260640 1 -- 172.21.15.98:0/3695023160 >> 172.21.15.98:0/3695023160 conn(0x7f6390074150 msgr2=0x7f6390074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:31.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.705+0000 7f6395260640 1 -- 172.21.15.98:0/3695023160 shutdown_connections 2024-09-10T20:59:31.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.705+0000 7f6395260640 1 -- 172.21.15.98:0/3695023160 wait complete. 2024-09-10T20:59:31.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.707+0000 7f6395260640 1 Processor -- start 2024-09-10T20:59:31.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.707+0000 7f6395260640 1 -- start start 2024-09-10T20:59:31.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f6395260640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f639006ca50 0x7f63900af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:31.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f6395260640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6390074c70 0x7f6390157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:31.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f6395260640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f639015bdb0 0x7f639015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:31.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f6395260640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f639006d290 con 0x7f6390074c70 2024-09-10T20:59:31.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f6395260640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f639006d110 con 0x7f639015bdb0 2024-09-10T20:59:31.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f6395260640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6390160660 con 0x7f639006ca50 2024-09-10T20:59:31.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f638e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6390074c70 0x7f6390157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:31.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f638e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6390074c70 0x7f6390157670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:52876/0 (socket says 172.21.15.98:52876) 2024-09-10T20:59:31.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f638e59c640 1 -- 172.21.15.98:0/564027878 learned_addr learned my addr 172.21.15.98:0/564027878 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:31.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f638f59e640 1 --2- 172.21.15.98:0/564027878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f639015bdb0 0x7f639015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:31.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f638ed9d640 1 --2- 172.21.15.98:0/564027878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f639006ca50 0x7f63900af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:31.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f638e59c640 1 -- 172.21.15.98:0/564027878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f639006ca50 msgr2=0x7f63900af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:31.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f638e59c640 1 --2- 172.21.15.98:0/564027878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f639006ca50 0x7f63900af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:31.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f638e59c640 1 -- 172.21.15.98:0/564027878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f639015bdb0 msgr2=0x7f639015eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:31.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f638e59c640 1 --2- 172.21.15.98:0/564027878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f639015bdb0 0x7f639015eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:31.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f638e59c640 1 -- 172.21.15.98:0/564027878 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639015c350 con 0x7f6390074c70 2024-09-10T20:59:31.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f638e59c640 1 --2- 172.21.15.98:0/564027878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6390074c70 0x7f6390157670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6384009950 tx=0x7f6384032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:31.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.708+0000 7f63777fe640 1 -- 172.21.15.98:0/564027878 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384040070 con 0x7f6390074c70 2024-09-10T20:59:31.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.709+0000 7f63777fe640 1 -- 172.21.15.98:0/564027878 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6384005310 con 0x7f6390074c70 2024-09-10T20:59:31.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.709+0000 7f6395260640 1 -- 172.21.15.98:0/564027878 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639015c5c0 con 0x7f6390074c70 2024-09-10T20:59:31.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.709+0000 7f63777fe640 1 -- 172.21.15.98:0/564027878 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63840047a0 con 0x7f6390074c70 2024-09-10T20:59:31.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.709+0000 7f6395260640 1 -- 172.21.15.98:0/564027878 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639015cb80 con 0x7f6390074c70 2024-09-10T20:59:31.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.709+0000 7f6395260640 1 -- 172.21.15.98:0/564027878 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639015b960 con 0x7f6390074c70 2024-09-10T20:59:31.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.710+0000 7f63777fe640 1 -- 172.21.15.98:0/564027878 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6384004940 con 0x7f6390074c70 2024-09-10T20:59:31.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.711+0000 7f63777fe640 1 --2- 172.21.15.98:0/564027878 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f637c077f70 0x7f637c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:31.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.711+0000 7f63777fe640 1 -- 172.21.15.98:0/564027878 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6362+0+0 (secure 0 0 0) 0x7f63840bb350 con 0x7f6390074c70 2024-09-10T20:59:31.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.711+0000 7f638ed9d640 1 --2- 172.21.15.98:0/564027878 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f637c077f70 0x7f637c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:31.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.711+0000 7f638ed9d640 1 --2- 172.21.15.98:0/564027878 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f637c077f70 0x7f637c07a430 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6380009b30 tx=0x7f6380005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:31.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.715+0000 7f63777fe640 1 -- 172.21.15.98:0/564027878 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f639015b960 con 0x7f6390074c70 2024-09-10T20:59:31.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:31.897+0000 7f6395260640 1 -- 172.21.15.98:0/564027878 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f63900b59f0 con 0x7f6390074c70 2024-09-10T20:59:32.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.591+0000 7f63777fe640 1 -- 172.21.15.98:0/564027878 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7f6384083580 con 0x7f6390074c70 2024-09-10T20:59:32.595 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-09-10T20:59:32.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.593+0000 7f6395260640 1 -- 172.21.15.98:0/564027878 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f637c077f70 msgr2=0x7f637c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:32.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.593+0000 7f6395260640 1 --2- 172.21.15.98:0/564027878 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f637c077f70 0x7f637c07a430 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6380009b30 tx=0x7f6380005f90 comp rx=0 tx=0).stop 2024-09-10T20:59:32.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.593+0000 7f6395260640 1 -- 172.21.15.98:0/564027878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6390074c70 msgr2=0x7f6390157670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:32.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.593+0000 7f6395260640 1 --2- 172.21.15.98:0/564027878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6390074c70 0x7f6390157670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6384009950 tx=0x7f6384032c10 comp rx=0 tx=0).stop 2024-09-10T20:59:32.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.593+0000 7f6395260640 1 -- 172.21.15.98:0/564027878 shutdown_connections 2024-09-10T20:59:32.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.594+0000 7f6395260640 1 --2- 172.21.15.98:0/564027878 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f637c077f70 0x7f637c07a430 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:32.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.594+0000 7f6395260640 1 --2- 172.21.15.98:0/564027878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f639015bdb0 0x7f639015eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:32.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.594+0000 7f6395260640 1 --2- 172.21.15.98:0/564027878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6390074c70 0x7f6390157670 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:32.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.594+0000 7f6395260640 1 --2- 172.21.15.98:0/564027878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f639006ca50 0x7f63900af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:32.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.594+0000 7f6395260640 1 -- 172.21.15.98:0/564027878 >> 172.21.15.98:0/564027878 conn(0x7f6390074150 msgr2=0x7f639006f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:32.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.594+0000 7f6395260640 1 -- 172.21.15.98:0/564027878 shutdown_connections 2024-09-10T20:59:32.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.594+0000 7f6395260640 1 -- 172.21.15.98:0/564027878 wait complete. 2024-09-10T20:59:32.611 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-10T20:59:32.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.718+0000 7fdf16471640 1 -- 172.21.15.98:0/426530694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf100ceb00 msgr2=0x7fdf100cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:32.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.718+0000 7fdf16471640 1 --2- 172.21.15.98:0/426530694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf100ceb00 0x7fdf100cef90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdf10076350 tx=0x7fdf0802a980 comp rx=0 tx=0).stop 2024-09-10T20:59:32.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.718+0000 7fdf16471640 1 -- 172.21.15.98:0/426530694 shutdown_connections 2024-09-10T20:59:32.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.718+0000 7fdf16471640 1 --2- 172.21.15.98:0/426530694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf100ceb00 0x7fdf100cef90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:32.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.718+0000 7fdf16471640 1 --2- 172.21.15.98:0/426530694 >> v2:172.21.15.190:3300/0 conn(0x7fdf10072680 0x7fdf10072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:32.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.718+0000 7fdf16471640 1 --2- 172.21.15.98:0/426530694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf10144780 0x7fdf100b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:32.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.718+0000 7fdf16471640 1 -- 172.21.15.98:0/426530694 >> 172.21.15.98:0/426530694 conn(0x7fdf100b1760 msgr2=0x7fdf100b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:32.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.718+0000 7fdf16471640 1 -- 172.21.15.98:0/426530694 shutdown_connections 2024-09-10T20:59:32.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.718+0000 7fdf16471640 1 -- 172.21.15.98:0/426530694 wait complete. 2024-09-10T20:59:32.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.719+0000 7fdf16471640 1 Processor -- start 2024-09-10T20:59:32.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.720+0000 7fdf16471640 1 -- start start 2024-09-10T20:59:32.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.720+0000 7fdf16471640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf10072680 0x7fdf10163ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:32.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.720+0000 7fdf16471640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf100ceb00 0x7fdf101592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:32.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.720+0000 7fdf16471640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf10144780 0x7fdf1006f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:32.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.720+0000 7fdf16471640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1015c8e0 con 0x7fdf10144780 2024-09-10T20:59:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.720+0000 7fdf16471640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1015c760 con 0x7fdf10072680 2024-09-10T20:59:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.720+0000 7fdf16471640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1015ca60 con 0x7fdf100ceb00 2024-09-10T20:59:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.720+0000 7fdf14a0e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf10144780 0x7fdf1006f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.720+0000 7fdf14a0e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf10144780 0x7fdf1006f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:52902/0 (socket says 172.21.15.98:52902) 2024-09-10T20:59:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.720+0000 7fdf14a0e640 1 -- 172.21.15.98:0/3672918176 learned_addr learned my addr 172.21.15.98:0/3672918176 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.720+0000 7fdf0ffff640 1 --2- 172.21.15.98:0/3672918176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf10072680 0x7fdf10163ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.720+0000 7fdf0f7fe640 1 --2- 172.21.15.98:0/3672918176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf100ceb00 0x7fdf101592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.721+0000 7fdf14a0e640 1 -- 172.21.15.98:0/3672918176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf100ceb00 msgr2=0x7fdf101592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.721+0000 7fdf14a0e640 1 --2- 172.21.15.98:0/3672918176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf100ceb00 0x7fdf101592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.721+0000 7fdf14a0e640 1 -- 172.21.15.98:0/3672918176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf10072680 msgr2=0x7fdf10163ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.721+0000 7fdf14a0e640 1 --2- 172.21.15.98:0/3672918176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf10072680 0x7fdf10163ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.721+0000 7fdf14a0e640 1 -- 172.21.15.98:0/3672918176 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf10159810 con 0x7fdf10144780 2024-09-10T20:59:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.721+0000 7fdf14a0e640 1 --2- 172.21.15.98:0/3672918176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf10144780 0x7fdf1006f330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdf0802af70 tx=0x7fdf08003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.721+0000 7fdf0cff9640 1 -- 172.21.15.98:0/3672918176 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08034050 con 0x7fdf10144780 2024-09-10T20:59:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.721+0000 7fdf0cff9640 1 -- 172.21.15.98:0/3672918176 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf08004550 con 0x7fdf10144780 2024-09-10T20:59:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.721+0000 7fdf16471640 1 -- 172.21.15.98:0/3672918176 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1013d480 con 0x7fdf10144780 2024-09-10T20:59:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.721+0000 7fdf0cff9640 1 -- 172.21.15.98:0/3672918176 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08043440 con 0x7fdf10144780 2024-09-10T20:59:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.721+0000 7fdf16471640 1 -- 172.21.15.98:0/3672918176 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf100702a0 con 0x7fdf10144780 2024-09-10T20:59:32.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.722+0000 7fdf16471640 1 -- 172.21.15.98:0/3672918176 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1015cf10 con 0x7fdf10144780 2024-09-10T20:59:32.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.723+0000 7fdf0cff9640 1 -- 172.21.15.98:0/3672918176 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fdf08041070 con 0x7fdf10144780 2024-09-10T20:59:32.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.723+0000 7fdf0cff9640 1 --2- 172.21.15.98:0/3672918176 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdefc077f70 0x7fdefc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:32.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.723+0000 7fdf0cff9640 1 -- 172.21.15.98:0/3672918176 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7fdf080bb100 con 0x7fdf10144780 2024-09-10T20:59:32.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.723+0000 7fdf0ffff640 1 --2- 172.21.15.98:0/3672918176 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdefc077f70 0x7fdefc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:32.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.724+0000 7fdf0ffff640 1 --2- 172.21.15.98:0/3672918176 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdefc077f70 0x7fdefc07a430 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdf000058c0 tx=0x7fdf00005f60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:32.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.727+0000 7fdf0cff9640 1 -- 172.21.15.98:0/3672918176 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fdf080835b0 con 0x7fdf10144780 2024-09-10T20:59:32.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:32.912+0000 7fdf16471640 1 -- 172.21.15.98:0/3672918176 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdf100b4d40 con 0x7fdf10144780 2024-09-10T20:59:33.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:33.595+0000 7fdf0cff9640 1 -- 172.21.15.98:0/3672918176 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7fdf08087fe0 con 0x7fdf10144780 2024-09-10T20:59:33.600 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data_ec' removed 2024-09-10T20:59:33.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:33.598+0000 7fdf16471640 1 -- 172.21.15.98:0/3672918176 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdefc077f70 msgr2=0x7fdefc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:33.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:33.598+0000 7fdf16471640 1 --2- 172.21.15.98:0/3672918176 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdefc077f70 0x7fdefc07a430 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdf000058c0 tx=0x7fdf00005f60 comp rx=0 tx=0).stop 2024-09-10T20:59:33.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:33.598+0000 7fdf16471640 1 -- 172.21.15.98:0/3672918176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf10144780 msgr2=0x7fdf1006f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:33.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:33.598+0000 7fdf16471640 1 --2- 172.21.15.98:0/3672918176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf10144780 0x7fdf1006f330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdf0802af70 tx=0x7fdf08003ef0 comp rx=0 tx=0).stop 2024-09-10T20:59:33.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:33.599+0000 7fdf16471640 1 -- 172.21.15.98:0/3672918176 shutdown_connections 2024-09-10T20:59:33.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:33.599+0000 7fdf16471640 1 --2- 172.21.15.98:0/3672918176 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdefc077f70 0x7fdefc07a430 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:33.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:33.599+0000 7fdf16471640 1 --2- 172.21.15.98:0/3672918176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf10144780 0x7fdf1006f330 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:33.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:33.599+0000 7fdf16471640 1 --2- 172.21.15.98:0/3672918176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf100ceb00 0x7fdf101592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:33.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:33.599+0000 7fdf16471640 1 --2- 172.21.15.98:0/3672918176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf10072680 0x7fdf10163ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:33.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:33.599+0000 7fdf16471640 1 -- 172.21.15.98:0/3672918176 >> 172.21.15.98:0/3672918176 conn(0x7fdf100b1760 msgr2=0x7fdf1015e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:33.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:33.600+0000 7fdf16471640 1 -- 172.21.15.98:0/3672918176 shutdown_connections 2024-09-10T20:59:33.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:33.600+0000 7fdf16471640 1 -- 172.21.15.98:0/3672918176 wait complete. 2024-09-10T20:59:33.618 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T20:59:33.618 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T20:59:33.619 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T20:59:33.619 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:59:33.619 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T20:59:33.619 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T20:59:33.619 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T20:59:33.619 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:59:33.619 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T20:59:33.620 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T20:59:33.620 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T20:59:33.620 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:59:33.620 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T20:59:33.620 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T20:59:33.620 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T20:59:33.620 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T20:59:33.629 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T20:59:33.630 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T20:59:33.631 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T20:59:33.631 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T20:59:33.632 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T20:59:33.632 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T20:59:33.672 INFO:tasks.ceph.mds.a:Started 2024-09-10T20:59:33.673 DEBUG:teuthology.parallel:result is None 2024-09-10T20:59:33.673 INFO:tasks.ceph.mds.c:Started 2024-09-10T20:59:33.673 DEBUG:teuthology.parallel:result is None 2024-09-10T20:59:33.675 INFO:tasks.ceph.mds.d:Started 2024-09-10T20:59:33.676 DEBUG:teuthology.parallel:result is None 2024-09-10T20:59:33.751 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-10T20:59:33.751 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-10T20:59:33.755 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2024-09-10T20:59:36.107 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-10T20:59:39.632 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T20:59:39.633 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T20:59:39.636 INFO:tasks.ceph.mds.b:Started 2024-09-10T20:59:39.636 DEBUG:teuthology.parallel:result is None 2024-09-10T20:59:39.637 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T20:59:39.719 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2024-09-10T20:59:39.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.748+0000 7f5154eed640 1 -- 172.21.15.98:0/2941758607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5150072680 msgr2=0x7f51500ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:39.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.748+0000 7f5154eed640 1 --2- 172.21.15.98:0/2941758607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5150072680 0x7f51500ceb00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f51500b2fc0 tx=0x7f513802a4d0 comp rx=0 tx=0).stop 2024-09-10T20:59:39.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.748+0000 7f5154eed640 1 -- 172.21.15.98:0/2941758607 shutdown_connections 2024-09-10T20:59:39.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.748+0000 7f5154eed640 1 --2- 172.21.15.98:0/2941758607 >> v2:172.21.15.190:3300/0 conn(0x7f5150072ad0 0x7f51500cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:39.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.748+0000 7f5154eed640 1 --2- 172.21.15.98:0/2941758607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5150072680 0x7f51500ceb00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:39.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.748+0000 7f5154eed640 1 -- 172.21.15.98:0/2941758607 >> 172.21.15.98:0/2941758607 conn(0x7f51500b1760 msgr2=0x7f51500b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:39.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.748+0000 7f5154eed640 1 -- 172.21.15.98:0/2941758607 shutdown_connections 2024-09-10T20:59:39.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.748+0000 7f5154eed640 1 -- 172.21.15.98:0/2941758607 wait complete. 2024-09-10T20:59:39.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.749+0000 7f5154eed640 1 Processor -- start 2024-09-10T20:59:39.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.750+0000 7f5154eed640 1 -- start start 2024-09-10T20:59:39.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.750+0000 7f5154eed640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5150072680 0x7f5150142500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:39.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.750+0000 7f5154eed640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5150072ad0 0x7f51501e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:39.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.750+0000 7f5154eed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5150144780 0x7f51501e9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:39.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.750+0000 7f5154eed640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f515015c8e0 con 0x7f5150072ad0 2024-09-10T20:59:39.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.750+0000 7f5154eed640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f515015c760 con 0x7f5150144780 2024-09-10T20:59:39.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.750+0000 7f5154eed640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f515015ca60 con 0x7f5150072680 2024-09-10T20:59:39.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.750+0000 7f514dd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5150072ad0 0x7f51501e8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:39.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.750+0000 7f514dd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5150072ad0 0x7f51501e8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:52938/0 (socket says 172.21.15.98:52938) 2024-09-10T20:59:39.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.750+0000 7f514dd9b640 1 -- 172.21.15.98:0/3759360365 learned_addr learned my addr 172.21.15.98:0/3759360365 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:39.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.751+0000 7f514e59c640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5150072680 0x7f5150142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:39.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.751+0000 7f514dd9b640 1 -- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5150072680 msgr2=0x7f5150142500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:39.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.751+0000 7f514ed9d640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5150144780 0x7f51501e9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:39.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.751+0000 7f514dd9b640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5150072680 0x7f5150142500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:39.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.751+0000 7f514dd9b640 1 -- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5150144780 msgr2=0x7f51501e9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:39.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.751+0000 7f514dd9b640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5150144780 0x7f51501e9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:39.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.751+0000 7f514dd9b640 1 -- 172.21.15.98:0/3759360365 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51501f81d0 con 0x7f5150072ad0 2024-09-10T20:59:39.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.751+0000 7f514e59c640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5150072680 0x7f5150142500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:39.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.751+0000 7f514dd9b640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5150072ad0 0x7f51501e8bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5138009850 tx=0x7f51380039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:39.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.751+0000 7f5146ffd640 1 -- 172.21.15.98:0/3759360365 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513803f070 con 0x7f5150072ad0 2024-09-10T20:59:39.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.751+0000 7f5146ffd640 1 -- 172.21.15.98:0/3759360365 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5138004890 con 0x7f5150072ad0 2024-09-10T20:59:39.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.751+0000 7f5154eed640 1 -- 172.21.15.98:0/3759360365 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51501f84b0 con 0x7f5150072ad0 2024-09-10T20:59:39.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.751+0000 7f5146ffd640 1 -- 172.21.15.98:0/3759360365 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138035030 con 0x7f5150072ad0 2024-09-10T20:59:39.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.751+0000 7f5154eed640 1 -- 172.21.15.98:0/3759360365 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51501f8a70 con 0x7f5150072ad0 2024-09-10T20:59:39.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.752+0000 7f5154eed640 1 -- 172.21.15.98:0/3759360365 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515015cf30 con 0x7f5150072ad0 2024-09-10T20:59:39.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.753+0000 7f5146ffd640 1 -- 172.21.15.98:0/3759360365 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5138004ca0 con 0x7f5150072ad0 2024-09-10T20:59:39.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.753+0000 7f5146ffd640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5130077f70 0x7f513007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:39.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.753+0000 7f5146ffd640 1 -- 172.21.15.98:0/3759360365 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5335+0+0 (secure 0 0 0) 0x7f51380bae50 con 0x7f5150072ad0 2024-09-10T20:59:39.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.753+0000 7f514e59c640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5130077f70 0x7f513007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:39.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.754+0000 7f514e59c640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5130077f70 0x7f513007a430 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f514000a810 tx=0x7f5140008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:39.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.757+0000 7f5146ffd640 1 -- 172.21.15.98:0/3759360365 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5138083b50 con 0x7f5150072ad0 2024-09-10T20:59:39.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:39.942+0000 7f5154eed640 1 -- 172.21.15.98:0/3759360365 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f51500b4d40 con 0x7f5150072ad0 2024-09-10T20:59:40.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:40.973+0000 7f5146ffd640 1 -- 172.21.15.98:0/3759360365 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v36) ==== 97+0+0 (secure 0 0 0) 0x7f5138083450 con 0x7f5150072ad0 2024-09-10T20:59:40.977 INFO:teuthology.orchestra.run.smithi098.stderr: removed all blocklist entries 2024-09-10T20:59:40.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:40.976+0000 7f5154eed640 1 -- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5130077f70 msgr2=0x7f513007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:40.976+0000 7f5154eed640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5130077f70 0x7f513007a430 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f514000a810 tx=0x7f5140008040 comp rx=0 tx=0).stop 2024-09-10T20:59:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:40.976+0000 7f5154eed640 1 -- 172.21.15.98:0/3759360365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5150072ad0 msgr2=0x7f51501e8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:40.976+0000 7f5154eed640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5150072ad0 0x7f51501e8bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5138009850 tx=0x7f51380039e0 comp rx=0 tx=0).stop 2024-09-10T20:59:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:40.976+0000 7f5154eed640 1 -- 172.21.15.98:0/3759360365 shutdown_connections 2024-09-10T20:59:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:40.976+0000 7f5154eed640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5130077f70 0x7f513007a430 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:40.976+0000 7f5154eed640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5150144780 0x7f51501e9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:40.976+0000 7f5154eed640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5150072ad0 0x7f51501e8bf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:40.976+0000 7f5154eed640 1 --2- 172.21.15.98:0/3759360365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5150072680 0x7f5150142500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:40.976+0000 7f5154eed640 1 -- 172.21.15.98:0/3759360365 >> 172.21.15.98:0/3759360365 conn(0x7f51500b1760 msgr2=0x7f51500702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:40.976+0000 7f5154eed640 1 -- 172.21.15.98:0/3759360365 shutdown_connections 2024-09-10T20:59:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:40.976+0000 7f5154eed640 1 -- 172.21.15.98:0/3759360365 wait complete. 2024-09-10T20:59:40.994 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T20:59:41.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.096+0000 7f8c30dcd640 1 -- 172.21.15.98:0/3236398086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c0cf3d0 msgr2=0x7f8c2c15c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:41.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.096+0000 7f8c30dcd640 1 --2- 172.21.15.98:0/3236398086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c0cf3d0 0x7f8c2c15c410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c1522f0 tx=0x7f8c1c02a410 comp rx=0 tx=0).stop 2024-09-10T20:59:41.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.097+0000 7f8c30dcd640 1 -- 172.21.15.98:0/3236398086 shutdown_connections 2024-09-10T20:59:41.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.097+0000 7f8c30dcd640 1 --2- 172.21.15.98:0/3236398086 >> v2:172.21.15.190:3300/0 conn(0x7f8c2c15b910 0x7f8c2c072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.097+0000 7f8c30dcd640 1 --2- 172.21.15.98:0/3236398086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c0cf3d0 0x7f8c2c15c410 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.097+0000 7f8c30dcd640 1 -- 172.21.15.98:0/3236398086 >> 172.21.15.98:0/3236398086 conn(0x7f8c2c0b1760 msgr2=0x7f8c2c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:41.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.097+0000 7f8c30dcd640 1 -- 172.21.15.98:0/3236398086 shutdown_connections 2024-09-10T20:59:41.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.097+0000 7f8c30dcd640 1 -- 172.21.15.98:0/3236398086 wait complete. 2024-09-10T20:59:41.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.098+0000 7f8c30dcd640 1 Processor -- start 2024-09-10T20:59:41.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.098+0000 7f8c30dcd640 1 -- start start 2024-09-10T20:59:41.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c30dcd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c2c072770 0x7f8c2c1f6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:41.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c30dcd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c2c0cf3d0 0x7f8c2c154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:41.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c30dcd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c15b910 0x7f8c2c154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:41.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c30dcd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c0b3ab0 con 0x7f8c2c15b910 2024-09-10T20:59:41.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c30dcd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c0b3930 con 0x7f8c2c072770 2024-09-10T20:59:41.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c30dcd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8c2c0b3c30 con 0x7f8c2c0cf3d0 2024-09-10T20:59:41.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c2ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c15b910 0x7f8c2c154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:41.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c2ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c15b910 0x7f8c2c154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45652/0 (socket says 172.21.15.98:45652) 2024-09-10T20:59:41.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c2ad9d640 1 -- 172.21.15.98:0/2029252529 learned_addr learned my addr 172.21.15.98:0/2029252529 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:41.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c2a59c640 1 --2- 172.21.15.98:0/2029252529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c2c072770 0x7f8c2c1f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:41.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c21d9b640 1 --2- 172.21.15.98:0/2029252529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c2c0cf3d0 0x7f8c2c154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:41.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c2ad9d640 1 -- 172.21.15.98:0/2029252529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c2c0cf3d0 msgr2=0x7f8c2c154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:41.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c2ad9d640 1 --2- 172.21.15.98:0/2029252529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c2c0cf3d0 0x7f8c2c154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c2ad9d640 1 -- 172.21.15.98:0/2029252529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c2c072770 msgr2=0x7f8c2c1f6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:41.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c2ad9d640 1 --2- 172.21.15.98:0/2029252529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c2c072770 0x7f8c2c1f6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.099+0000 7f8c2ad9d640 1 -- 172.21.15.98:0/2029252529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2c13f130 con 0x7f8c2c15b910 2024-09-10T20:59:41.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.100+0000 7f8c2ad9d640 1 --2- 172.21.15.98:0/2029252529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c15b910 0x7f8c2c154f90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400e990 tx=0x7f8c2400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:41.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.100+0000 7f8c237fe640 1 -- 172.21.15.98:0/2029252529 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24004d20 con 0x7f8c2c15b910 2024-09-10T20:59:41.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.100+0000 7f8c237fe640 1 -- 172.21.15.98:0/2029252529 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c2400cb00 con 0x7f8c2c15b910 2024-09-10T20:59:41.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.100+0000 7f8c30dcd640 1 -- 172.21.15.98:0/2029252529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c2c13f3e0 con 0x7f8c2c15b910 2024-09-10T20:59:41.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.100+0000 7f8c237fe640 1 -- 172.21.15.98:0/2029252529 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24005020 con 0x7f8c2c15b910 2024-09-10T20:59:41.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.100+0000 7f8c30dcd640 1 -- 172.21.15.98:0/2029252529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c2c13f9a0 con 0x7f8c2c15b910 2024-09-10T20:59:41.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.101+0000 7f8c30dcd640 1 -- 172.21.15.98:0/2029252529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c2c0b4150 con 0x7f8c2c15b910 2024-09-10T20:59:41.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.101+0000 7f8c237fe640 1 -- 172.21.15.98:0/2029252529 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8c2400f040 con 0x7f8c2c15b910 2024-09-10T20:59:41.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.102+0000 7f8c237fe640 1 --2- 172.21.15.98:0/2029252529 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8c0c077f70 0x7f8c0c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:41.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.102+0000 7f8c237fe640 1 -- 172.21.15.98:0/2029252529 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f8c2409a1b0 con 0x7f8c2c15b910 2024-09-10T20:59:41.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.102+0000 7f8c2a59c640 1 --2- 172.21.15.98:0/2029252529 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8c0c077f70 0x7f8c0c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:41.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.103+0000 7f8c2a59c640 1 --2- 172.21.15.98:0/2029252529 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8c0c077f70 0x7f8c0c07a430 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c034000 tx=0x7f8c1c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:41.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.107+0000 7f8c237fe640 1 -- 172.21.15.98:0/2029252529 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8c24016030 con 0x7f8c2c15b910 2024-09-10T20:59:41.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.345+0000 7f8c30dcd640 1 -- 172.21.15.98:0/2029252529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8c2c154610 con 0x7f8c2c15b910 2024-09-10T20:59:41.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.346+0000 7f8c237fe640 1 -- 172.21.15.98:0/2029252529 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f8c240627a0 con 0x7f8c2c15b910 2024-09-10T20:59:41.350 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:41.350 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-10T20:59:41.350 INFO:teuthology.orchestra.run.smithi098.stdout: "auth_dump": [ 2024-09-10T20:59:41.350 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.350 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.a", 2024-09-10T20:59:41.350 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDKsuBmDQc7IhAAfVGGH67CSTDxtnhwsoVZjg==", 2024-09-10T20:59:41.350 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.350 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-10T20:59:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-10T20:59:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-10T20:59:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.b", 2024-09-10T20:59:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDKsuBmjXnsLRAAmoNy11LPHYxA1o9GX7i90g==", 2024-09-10T20:59:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-10T20:59:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-10T20:59:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-10T20:59:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.351 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.c", 2024-09-10T20:59:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDKsuBmIjk8KRAAM5ZHLlzSDFDzwax1tqAuuw==", 2024-09-10T20:59:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-10T20:59:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-10T20:59:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-10T20:59:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.352 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.d", 2024-09-10T20:59:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDKsuBmi9PnNBAACkF8eU6DQLO6lfgg/y7BWw==", 2024-09-10T20:59:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-10T20:59:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-10T20:59:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-10T20:59:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.0", 2024-09-10T20:59:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDOsuBmsYv0FxAAxF0+GnRAMDFdUFA8NqVuPA==", 2024-09-10T20:59:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-10T20:59:41.353 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-10T20:59:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.1", 2024-09-10T20:59:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDRsuBmaiMRKBAAOI/EtX7NA6eaeNwOC/VptA==", 2024-09-10T20:59:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-10T20:59:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-10T20:59:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.354 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.2", 2024-09-10T20:59:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDVsuBmDVVMCxAACm/EhpctxJidYZRHKTksCw==", 2024-09-10T20:59:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-10T20:59:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-10T20:59:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.3", 2024-09-10T20:59:41.355 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDYsuBmFv9jHBAAGwAe5pgbvheRQaGOgEM70A==", 2024-09-10T20:59:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-10T20:59:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-10T20:59:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.4", 2024-09-10T20:59:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDesuBmvjnNMhAAxSyqvd3hcOndQZqE7b9Mug==", 2024-09-10T20:59:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-10T20:59:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-10T20:59:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.356 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.5", 2024-09-10T20:59:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDisuBmdlELFhAASuvCbhdNG/3eLu8+MUnMpw==", 2024-09-10T20:59:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-10T20:59:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-10T20:59:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.6", 2024-09-10T20:59:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDlsuBmwZ2UFxAA+kl84axYER+Zf74///EPtA==", 2024-09-10T20:59:41.357 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-10T20:59:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-10T20:59:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.7", 2024-09-10T20:59:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDosuBmJD0QKBAA/LnYMX4Yt9Y4qgJs6sYErg==", 2024-09-10T20:59:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-10T20:59:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-10T20:59:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.358 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.0", 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDKsuBm9hT3OBAAecwpOYTkz+JIO0Xg/GolGw==", 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.1", 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDLsuBmfGwbBhAA9wRJAg8fHtVSoExyGmo3bw==", 2024-09-10T20:59:41.359 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-10T20:59:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-10T20:59:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-10T20:59:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-10T20:59:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.2", 2024-09-10T20:59:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDLsuBmoKLnAhAArP2MzQowgxjBwOob6Yd0/Q==", 2024-09-10T20:59:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-10T20:59:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-10T20:59:41.360 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-10T20:59:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-10T20:59:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.3", 2024-09-10T20:59:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDLsuBmJoCtCxAAYcwdH65t1Px4BeLbOa15fw==", 2024-09-10T20:59:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-10T20:59:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-10T20:59:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-10T20:59:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-10T20:59:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.admin", 2024-09-10T20:59:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDKsuBmwVCsBhAA6xJPmpbUT3cRI4aPJVCx8Q==", 2024-09-10T20:59:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-10T20:59:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-10T20:59:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-10T20:59:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-mds", 2024-09-10T20:59:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDusuBmqy5xIBAA9SUzprHYr5v0qQ0LHPNCdw==", 2024-09-10T20:59:41.362 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-mds" 2024-09-10T20:59:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-mgr", 2024-09-10T20:59:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDusuBmuTZxIBAAzGuc8oto2jecZi/VrnbKaw==", 2024-09-10T20:59:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-10T20:59:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-osd", 2024-09-10T20:59:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDusuBmcj5xIBAAyHNr1NtyiDwNc+rVNfCn9w==", 2024-09-10T20:59:41.363 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-osd" 2024-09-10T20:59:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-rbd", 2024-09-10T20:59:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDusuBm+URxIBAA9SdSd7a/oi79LyoQIa6L2A==", 2024-09-10T20:59:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-10T20:59:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-10T20:59:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDusuBmjUtxIBAAVY7391j8DmOy7DZgBW0WgQ==", 2024-09-10T20:59:41.364 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-10T20:59:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-rgw", 2024-09-10T20:59:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDusuBmXVJxIBAAHfw650Kf/+QvGN7AyWBVfQ==", 2024-09-10T20:59:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-10T20:59:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.x", 2024-09-10T20:59:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDKsuBm7OUOHxAANoyF+Xxo2J5mgEShMqV/Rw==", 2024-09-10T20:59:41.365 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-10T20:59:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-10T20:59:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-10T20:59:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.y", 2024-09-10T20:59:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDKsuBme6kzGxAAcvRmmGzk/xeEYu9MHDpypg==", 2024-09-10T20:59:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-10T20:59:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-10T20:59:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-10T20:59:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-10T20:59:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.366 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:41.367 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-10T20:59:41.367 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-10T20:59:41.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.348+0000 7f8c30dcd640 1 -- 172.21.15.98:0/2029252529 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8c0c077f70 msgr2=0x7f8c0c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:41.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.348+0000 7f8c30dcd640 1 --2- 172.21.15.98:0/2029252529 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8c0c077f70 0x7f8c0c07a430 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c034000 tx=0x7f8c1c004730 comp rx=0 tx=0).stop 2024-09-10T20:59:41.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.348+0000 7f8c30dcd640 1 -- 172.21.15.98:0/2029252529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c15b910 msgr2=0x7f8c2c154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:41.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.348+0000 7f8c30dcd640 1 --2- 172.21.15.98:0/2029252529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c15b910 0x7f8c2c154f90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400e990 tx=0x7f8c2400ee50 comp rx=0 tx=0).stop 2024-09-10T20:59:41.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.348+0000 7f8c30dcd640 1 -- 172.21.15.98:0/2029252529 shutdown_connections 2024-09-10T20:59:41.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.348+0000 7f8c30dcd640 1 --2- 172.21.15.98:0/2029252529 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8c0c077f70 0x7f8c0c07a430 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.348+0000 7f8c30dcd640 1 --2- 172.21.15.98:0/2029252529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c15b910 0x7f8c2c154f90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.348+0000 7f8c30dcd640 1 --2- 172.21.15.98:0/2029252529 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c2c0cf3d0 0x7f8c2c154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.348+0000 7f8c30dcd640 1 --2- 172.21.15.98:0/2029252529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c2c072770 0x7f8c2c1f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.348+0000 7f8c30dcd640 1 -- 172.21.15.98:0/2029252529 >> 172.21.15.98:0/2029252529 conn(0x7f8c2c0b1760 msgr2=0x7f8c2c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:41.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.348+0000 7f8c30dcd640 1 -- 172.21.15.98:0/2029252529 shutdown_connections 2024-09-10T20:59:41.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.348+0000 7f8c30dcd640 1 -- 172.21.15.98:0/2029252529 wait complete. 2024-09-10T20:59:41.369 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-10T20:59:41.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.467+0000 7ff4884bd640 1 -- 172.21.15.98:0/861623565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff48015e060 msgr2=0x7ff4800b6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:41.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.467+0000 7ff4884bd640 1 --2- 172.21.15.98:0/861623565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff48015e060 0x7ff4800b6dc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff474009830 tx=0x7ff47402a5c0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.467+0000 7ff4884bd640 1 -- 172.21.15.98:0/861623565 shutdown_connections 2024-09-10T20:59:41.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.467+0000 7ff4884bd640 1 --2- 172.21.15.98:0/861623565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff48015e060 0x7ff4800b6dc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.467+0000 7ff4884bd640 1 -- 172.21.15.98:0/861623565 >> 172.21.15.98:0/861623565 conn(0x7ff4800d3f70 msgr2=0x7ff4800d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:41.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.467+0000 7ff4884bd640 1 -- 172.21.15.98:0/861623565 shutdown_connections 2024-09-10T20:59:41.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.467+0000 7ff4884bd640 1 -- 172.21.15.98:0/861623565 wait complete. 2024-09-10T20:59:41.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.469+0000 7ff4884bd640 1 Processor -- start 2024-09-10T20:59:41.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.469+0000 7ff4884bd640 1 -- start start 2024-09-10T20:59:41.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.469+0000 7ff4884bd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4800d4af0 0x7ff48015f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:41.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.469+0000 7ff4884bd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff48015e060 0x7ff4801f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:41.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.469+0000 7ff4884bd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4800c92b0 0x7ff480152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:41.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.469+0000 7ff4884bd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff4800cf150 con 0x7ff4800d4af0 2024-09-10T20:59:41.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.469+0000 7ff4884bd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff4800d2eb0 con 0x7ff48015e060 2024-09-10T20:59:41.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.469+0000 7ff4884bd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff4800cf2d0 con 0x7ff4800c92b0 2024-09-10T20:59:41.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.469+0000 7ff486259640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4800d4af0 0x7ff48015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:41.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.469+0000 7ff486259640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4800d4af0 0x7ff48015f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45666/0 (socket says 172.21.15.98:45666) 2024-09-10T20:59:41.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.469+0000 7ff486259640 1 -- 172.21.15.98:0/1048617410 learned_addr learned my addr 172.21.15.98:0/1048617410 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:41.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.470+0000 7ff485a58640 1 --2- 172.21.15.98:0/1048617410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff48015e060 0x7ff4801f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:41.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.470+0000 7ff486a5a640 1 --2- 172.21.15.98:0/1048617410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4800c92b0 0x7ff480152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:41.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.470+0000 7ff486259640 1 -- 172.21.15.98:0/1048617410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4800c92b0 msgr2=0x7ff480152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:41.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.470+0000 7ff486259640 1 --2- 172.21.15.98:0/1048617410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4800c92b0 0x7ff480152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.470+0000 7ff486259640 1 -- 172.21.15.98:0/1048617410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff48015e060 msgr2=0x7ff4801f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:41.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.470+0000 7ff486259640 1 --2- 172.21.15.98:0/1048617410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff48015e060 0x7ff4801f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.470+0000 7ff486259640 1 -- 172.21.15.98:0/1048617410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48015a8c0 con 0x7ff4800d4af0 2024-09-10T20:59:41.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.470+0000 7ff486259640 1 --2- 172.21.15.98:0/1048617410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4800d4af0 0x7ff48015f150 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff474009950 tx=0x7ff474033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:41.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.470+0000 7ff472ffd640 1 -- 172.21.15.98:0/1048617410 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474031d60 con 0x7ff4800d4af0 2024-09-10T20:59:41.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.470+0000 7ff472ffd640 1 -- 172.21.15.98:0/1048617410 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff474005020 con 0x7ff4800d4af0 2024-09-10T20:59:41.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.470+0000 7ff472ffd640 1 -- 172.21.15.98:0/1048617410 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff474005470 con 0x7ff4800d4af0 2024-09-10T20:59:41.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.470+0000 7ff4884bd640 1 -- 172.21.15.98:0/1048617410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48015aba0 con 0x7ff4800d4af0 2024-09-10T20:59:41.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.470+0000 7ff4884bd640 1 -- 172.21.15.98:0/1048617410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48015ec30 con 0x7ff4800d4af0 2024-09-10T20:59:41.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.471+0000 7ff4884bd640 1 -- 172.21.15.98:0/1048617410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff48015e780 con 0x7ff4800d4af0 2024-09-10T20:59:41.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.472+0000 7ff472ffd640 1 -- 172.21.15.98:0/1048617410 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff4740047a0 con 0x7ff4800d4af0 2024-09-10T20:59:41.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.472+0000 7ff472ffd640 1 --2- 172.21.15.98:0/1048617410 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff464077f70 0x7ff46407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:41.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.472+0000 7ff472ffd640 1 -- 172.21.15.98:0/1048617410 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7ff4740bad30 con 0x7ff4800d4af0 2024-09-10T20:59:41.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.472+0000 7ff485a58640 1 --2- 172.21.15.98:0/1048617410 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff464077f70 0x7ff46407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:41.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.473+0000 7ff485a58640 1 --2- 172.21.15.98:0/1048617410 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff464077f70 0x7ff46407a430 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff47c000f60 tx=0x7ff47c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:41.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.476+0000 7ff472ffd640 1 -- 172.21.15.98:0/1048617410 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff47403f030 con 0x7ff4800d4af0 2024-09-10T20:59:41.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.716+0000 7ff4884bd640 1 -- 172.21.15.98:0/1048617410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7ff4800cf780 con 0x7ff4800d4af0 2024-09-10T20:59:41.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.717+0000 7ff472ffd640 1 -- 172.21.15.98:0/1048617410 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7ff4740839f0 con 0x7ff4800d4af0 2024-09-10T20:59:41.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.720+0000 7ff4884bd640 1 -- 172.21.15.98:0/1048617410 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff464077f70 msgr2=0x7ff46407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:41.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.720+0000 7ff4884bd640 1 --2- 172.21.15.98:0/1048617410 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff464077f70 0x7ff46407a430 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff47c000f60 tx=0x7ff47c009040 comp rx=0 tx=0).stop 2024-09-10T20:59:41.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.720+0000 7ff4884bd640 1 -- 172.21.15.98:0/1048617410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4800d4af0 msgr2=0x7ff48015f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:41.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.720+0000 7ff4884bd640 1 --2- 172.21.15.98:0/1048617410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4800d4af0 0x7ff48015f150 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff474009950 tx=0x7ff474033ce0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.720+0000 7ff4884bd640 1 -- 172.21.15.98:0/1048617410 shutdown_connections 2024-09-10T20:59:41.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.720+0000 7ff4884bd640 1 --2- 172.21.15.98:0/1048617410 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff464077f70 0x7ff46407a430 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.720+0000 7ff4884bd640 1 --2- 172.21.15.98:0/1048617410 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4800c92b0 0x7ff480152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.720+0000 7ff4884bd640 1 --2- 172.21.15.98:0/1048617410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff48015e060 0x7ff4801f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.720+0000 7ff4884bd640 1 --2- 172.21.15.98:0/1048617410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4800d4af0 0x7ff48015f150 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.720+0000 7ff4884bd640 1 -- 172.21.15.98:0/1048617410 >> 172.21.15.98:0/1048617410 conn(0x7ff4800d3f70 msgr2=0x7ff4800b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:41.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.720+0000 7ff4884bd640 1 -- 172.21.15.98:0/1048617410 shutdown_connections 2024-09-10T20:59:41.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.720+0000 7ff4884bd640 1 -- 172.21.15.98:0/1048617410 wait complete. 2024-09-10T20:59:41.735 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-10T20:59:41.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.831+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1295322969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ed815be00 msgr2=0x7f1ed8070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:41.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.831+0000 7f1edd3c9640 1 --2- 172.21.15.98:0/1295322969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ed815be00 0x7f1ed8070760 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8075550 tx=0x7f1ec802a560 comp rx=0 tx=0).stop 2024-09-10T20:59:41.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.831+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1295322969 shutdown_connections 2024-09-10T20:59:41.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.831+0000 7f1edd3c9640 1 --2- 172.21.15.98:0/1295322969 >> v2:172.21.15.190:3300/0 conn(0x7f1ed80d2830 0x7f1ed80cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.831+0000 7f1edd3c9640 1 --2- 172.21.15.98:0/1295322969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ed815be00 0x7f1ed8070760 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.831+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1295322969 >> 172.21.15.98:0/1295322969 conn(0x7f1ed80d1d10 msgr2=0x7f1ed80d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:41.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.831+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1295322969 shutdown_connections 2024-09-10T20:59:41.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.831+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1295322969 wait complete. 2024-09-10T20:59:41.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.833+0000 7f1edd3c9640 1 Processor -- start 2024-09-10T20:59:41.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.833+0000 7f1edd3c9640 1 -- start start 2024-09-10T20:59:41.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.833+0000 7f1edd3c9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ed80d02f0 0x7f1ed80c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:41.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.833+0000 7f1edd3c9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ed80d2830 0x7f1ed80b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:41.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.833+0000 7f1edd3c9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ed815be00 0x7f1ed81f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:41.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.833+0000 7f1edd3c9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed80cd1d0 con 0x7f1ed815be00 2024-09-10T20:59:41.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.833+0000 7f1edd3c9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed80cd050 con 0x7f1ed80d2830 2024-09-10T20:59:41.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.833+0000 7f1edd3c9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed80cd350 con 0x7f1ed80d02f0 2024-09-10T20:59:41.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.833+0000 7f1ed77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ed815be00 0x7f1ed81f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:41.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.833+0000 7f1ed77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ed815be00 0x7f1ed81f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45680/0 (socket says 172.21.15.98:45680) 2024-09-10T20:59:41.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.833+0000 7f1ed77fe640 1 -- 172.21.15.98:0/1630793776 learned_addr learned my addr 172.21.15.98:0/1630793776 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:41.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.834+0000 7f1ed6ffd640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ed80d02f0 0x7f1ed80c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:41.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.834+0000 7f1ed67fc640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ed80d2830 0x7f1ed80b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:41.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.834+0000 7f1ed77fe640 1 -- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ed80d02f0 msgr2=0x7f1ed80c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:41.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.834+0000 7f1ed77fe640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ed80d02f0 0x7f1ed80c5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.834+0000 7f1ed77fe640 1 -- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ed80d2830 msgr2=0x7f1ed80b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:41.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.834+0000 7f1ed77fe640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ed80d2830 0x7f1ed80b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:41.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.834+0000 7f1ed77fe640 1 -- 172.21.15.98:0/1630793776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed80b7900 con 0x7f1ed815be00 2024-09-10T20:59:41.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.834+0000 7f1ed6ffd640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ed80d02f0 0x7f1ed80c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:41.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.834+0000 7f1ed77fe640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ed815be00 0x7f1ed81f61e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000c780 tx=0x7f1ed000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:41.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.834+0000 7f1ecf7fe640 1 -- 172.21.15.98:0/1630793776 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0017070 con 0x7f1ed815be00 2024-09-10T20:59:41.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.834+0000 7f1ecf7fe640 1 -- 172.21.15.98:0/1630793776 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ed0003c60 con 0x7f1ed815be00 2024-09-10T20:59:41.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.834+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1630793776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed80c8450 con 0x7f1ed815be00 2024-09-10T20:59:41.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.834+0000 7f1ecf7fe640 1 -- 172.21.15.98:0/1630793776 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00052d0 con 0x7f1ed815be00 2024-09-10T20:59:41.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.834+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1630793776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed80b50c0 con 0x7f1ed815be00 2024-09-10T20:59:41.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.835+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1630793776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed80cf8e0 con 0x7f1ed815be00 2024-09-10T20:59:41.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.836+0000 7f1ecf7fe640 1 -- 172.21.15.98:0/1630793776 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1ed0005470 con 0x7f1ed815be00 2024-09-10T20:59:41.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.836+0000 7f1ecf7fe640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1eb8077f70 0x7f1eb807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:41.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.836+0000 7f1ecf7fe640 1 -- 172.21.15.98:0/1630793776 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f1ed00994c0 con 0x7f1ed815be00 2024-09-10T20:59:41.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.836+0000 7f1ed6ffd640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1eb8077f70 0x7f1eb807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:41.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.837+0000 7f1ed6ffd640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1eb8077f70 0x7f1eb807a430 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8033000 tx=0x7f1ec8004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:41.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:41.840+0000 7f1ecf7fe640 1 -- 172.21.15.98:0/1630793776 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1ed0062180 con 0x7f1ed815be00 2024-09-10T20:59:42.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.081+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1630793776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f1ed80b7c90 con 0x7f1ed815be00 2024-09-10T20:59:42.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.083+0000 7f1ecf7fe640 1 -- 172.21.15.98:0/1630793776 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f1ed0061a80 con 0x7f1ed815be00 2024-09-10T20:59:42.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.085+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1eb8077f70 msgr2=0x7f1eb807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.085+0000 7f1edd3c9640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1eb8077f70 0x7f1eb807a430 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8033000 tx=0x7f1ec8004730 comp rx=0 tx=0).stop 2024-09-10T20:59:42.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.085+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1630793776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ed815be00 msgr2=0x7f1ed81f61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.085+0000 7f1edd3c9640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ed815be00 0x7f1ed81f61e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000c780 tx=0x7f1ed000cb80 comp rx=0 tx=0).stop 2024-09-10T20:59:42.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.085+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1630793776 shutdown_connections 2024-09-10T20:59:42.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.085+0000 7f1edd3c9640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1eb8077f70 0x7f1eb807a430 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.085+0000 7f1edd3c9640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ed815be00 0x7f1ed81f61e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.085+0000 7f1edd3c9640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ed80d2830 0x7f1ed80b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.085+0000 7f1edd3c9640 1 --2- 172.21.15.98:0/1630793776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ed80d02f0 0x7f1ed80c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.085+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1630793776 >> 172.21.15.98:0/1630793776 conn(0x7f1ed80d1d10 msgr2=0x7f1ed80b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:42.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.085+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1630793776 shutdown_connections 2024-09-10T20:59:42.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.085+0000 7f1edd3c9640 1 -- 172.21.15.98:0/1630793776 wait complete. 2024-09-10T20:59:42.101 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-10T20:59:42.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.198+0000 7f63cd3cd640 1 -- 172.21.15.98:0/1881809196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b0004160 msgr2=0x7f63b0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.198+0000 7f63cd3cd640 1 --2- 172.21.15.98:0/1881809196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63b0004160 0x7f63b0004800 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00ab10 tx=0x7f63b0005830 comp rx=0 tx=0).stop 2024-09-10T20:59:42.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.198+0000 7f63c77fe640 1 -- 172.21.15.98:0/1881809196 reap_dead start 2024-09-10T20:59:42.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.198+0000 7f63cd3cd640 1 -- 172.21.15.98:0/1881809196 shutdown_connections 2024-09-10T20:59:42.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.198+0000 7f63cd3cd640 1 -- 172.21.15.98:0/1881809196 >> 172.21.15.98:0/1881809196 conn(0x7f63c809d170 msgr2=0x7f63c80d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:42.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.199+0000 7f63cd3cd640 1 -- 172.21.15.98:0/1881809196 shutdown_connections 2024-09-10T20:59:42.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.199+0000 7f63cd3cd640 1 -- 172.21.15.98:0/1881809196 wait complete. 2024-09-10T20:59:42.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.200+0000 7f63cd3cd640 1 Processor -- start 2024-09-10T20:59:42.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.200+0000 7f63cd3cd640 1 -- start start 2024-09-10T20:59:42.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63cd3cd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b0004160 0x7f63c80cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:42.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63cd3cd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63c8157480 0x7f63c80b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:42.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63cd3cd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c80b88a0 0x7f63c80b8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:42.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63cd3cd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f63c80b4bc0 con 0x7f63c80b88a0 2024-09-10T20:59:42.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63cd3cd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f63c80b4a40 con 0x7f63b0004160 2024-09-10T20:59:42.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63cd3cd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f63c80b4d40 con 0x7f63c8157480 2024-09-10T20:59:42.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63c77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c80b88a0 0x7f63c80b8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:42.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63c77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c80b88a0 0x7f63c80b8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45686/0 (socket says 172.21.15.98:45686) 2024-09-10T20:59:42.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63c77fe640 1 -- 172.21.15.98:0/2881806642 learned_addr learned my addr 172.21.15.98:0/2881806642 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:42.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63c6ffd640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b0004160 0x7f63c80cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:42.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63c67fc640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63c8157480 0x7f63c80b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:42.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63c77fe640 1 -- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63c8157480 msgr2=0x7f63c80b6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63c77fe640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63c8157480 0x7f63c80b6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63c77fe640 1 -- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b0004160 msgr2=0x7f63c80cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63c77fe640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b0004160 0x7f63c80cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63c77fe640 1 -- 172.21.15.98:0/2881806642 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c81e9250 con 0x7f63c80b88a0 2024-09-10T20:59:42.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63c6ffd640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b0004160 0x7f63c80cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:42.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63c77fe640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c80b88a0 0x7f63c80b8d00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f63c80b54f0 tx=0x7f63c0006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:42.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63af7fe640 1 -- 172.21.15.98:0/2881806642 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c000fa00 con 0x7f63c80b88a0 2024-09-10T20:59:42.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63af7fe640 1 -- 172.21.15.98:0/2881806642 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63c000fba0 con 0x7f63c80b88a0 2024-09-10T20:59:42.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63cd3cd640 1 -- 172.21.15.98:0/2881806642 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c80cb2b0 con 0x7f63c80b88a0 2024-09-10T20:59:42.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63af7fe640 1 -- 172.21.15.98:0/2881806642 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0011430 con 0x7f63c80b88a0 2024-09-10T20:59:42.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.201+0000 7f63cd3cd640 1 -- 172.21.15.98:0/2881806642 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c81e8c90 con 0x7f63c80b88a0 2024-09-10T20:59:42.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.202+0000 7f63cd3cd640 1 -- 172.21.15.98:0/2881806642 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c80d26a0 con 0x7f63c80b88a0 2024-09-10T20:59:42.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.203+0000 7f63af7fe640 1 -- 172.21.15.98:0/2881806642 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f63c00115d0 con 0x7f63c80b88a0 2024-09-10T20:59:42.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.203+0000 7f63af7fe640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f63b4077f70 0x7f63b407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:42.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.204+0000 7f63af7fe640 1 -- 172.21.15.98:0/2881806642 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f63c009a550 con 0x7f63c80b88a0 2024-09-10T20:59:42.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.204+0000 7f63c6ffd640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f63b4077f70 0x7f63b407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:42.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.204+0000 7f63c6ffd640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f63b4077f70 0x7f63b407a430 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f63b800ab60 tx=0x7f63b8007040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:42.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.208+0000 7f63af7fe640 1 -- 172.21.15.98:0/2881806642 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f63c00632a0 con 0x7f63c80b88a0 2024-09-10T20:59:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.449+0000 7f63cd3cd640 1 -- 172.21.15.98:0/2881806642 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f63c80cec80 con 0x7f63c80b88a0 2024-09-10T20:59:42.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.452+0000 7f63af7fe640 1 -- 172.21.15.98:0/2881806642 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f63c0062ba0 con 0x7f63c80b88a0 2024-09-10T20:59:42.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.454+0000 7f63cd3cd640 1 -- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f63b4077f70 msgr2=0x7f63b407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.454+0000 7f63cd3cd640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f63b4077f70 0x7f63b407a430 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f63b800ab60 tx=0x7f63b8007040 comp rx=0 tx=0).stop 2024-09-10T20:59:42.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.454+0000 7f63cd3cd640 1 -- 172.21.15.98:0/2881806642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c80b88a0 msgr2=0x7f63c80b8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.454+0000 7f63cd3cd640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c80b88a0 0x7f63c80b8d00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f63c80b54f0 tx=0x7f63c0006720 comp rx=0 tx=0).stop 2024-09-10T20:59:42.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.454+0000 7f63cd3cd640 1 -- 172.21.15.98:0/2881806642 shutdown_connections 2024-09-10T20:59:42.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.454+0000 7f63cd3cd640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f63b4077f70 0x7f63b407a430 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.454+0000 7f63cd3cd640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63c80b88a0 0x7f63c80b8d00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.454+0000 7f63cd3cd640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63c8157480 0x7f63c80b6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.454+0000 7f63cd3cd640 1 --2- 172.21.15.98:0/2881806642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63b0004160 0x7f63c80cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.454+0000 7f63cd3cd640 1 -- 172.21.15.98:0/2881806642 >> 172.21.15.98:0/2881806642 conn(0x7f63c809d170 msgr2=0x7f63c80cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:42.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.454+0000 7f63cd3cd640 1 -- 172.21.15.98:0/2881806642 shutdown_connections 2024-09-10T20:59:42.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.454+0000 7f63cd3cd640 1 -- 172.21.15.98:0/2881806642 wait complete. 2024-09-10T20:59:42.470 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-10T20:59:42.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.567+0000 7f0f22e2a640 1 -- 172.21.15.98:0/3020567929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f04004800 msgr2=0x7f0f04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.567+0000 7f0f22e2a640 1 --2- 172.21.15.98:0/3020567929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f04004800 0x7f0f04004c80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00c260 tx=0x7f0f04005830 comp rx=0 tx=0).stop 2024-09-10T20:59:42.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.567+0000 7f0f22e2a640 1 -- 172.21.15.98:0/3020567929 shutdown_connections 2024-09-10T20:59:42.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.567+0000 7f0f22e2a640 1 --2- 172.21.15.98:0/3020567929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f04004800 0x7f0f04004c80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.567+0000 7f0f22e2a640 1 -- 172.21.15.98:0/3020567929 >> 172.21.15.98:0/3020567929 conn(0x7f0f1c0d0c00 msgr2=0x7f0f1c0d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:42.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.567+0000 7f0f22e2a640 1 -- 172.21.15.98:0/3020567929 shutdown_connections 2024-09-10T20:59:42.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.568+0000 7f0f22e2a640 1 -- 172.21.15.98:0/3020567929 wait complete. 2024-09-10T20:59:42.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.569+0000 7f0f22e2a640 1 Processor -- start 2024-09-10T20:59:42.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.569+0000 7f0f22e2a640 1 -- start start 2024-09-10T20:59:42.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f22e2a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f04004800 0x7f0f1c14a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:42.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f22e2a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f1c14a040 0x7f0f1c14fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:42.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f22e2a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c154210 0x7f0f1c1546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:42.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f22e2a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c0b4180 con 0x7f0f04004800 2024-09-10T20:59:42.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f22e2a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c0b4000 con 0x7f0f1c14a040 2024-09-10T20:59:42.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f22e2a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0f1c0b4300 con 0x7f0f1c154210 2024-09-10T20:59:42.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f20bc6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f04004800 0x7f0f1c14a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:42.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f20bc6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f04004800 0x7f0f1c14a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45714/0 (socket says 172.21.15.98:45714) 2024-09-10T20:59:42.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f20bc6640 1 -- 172.21.15.98:0/1868202160 learned_addr learned my addr 172.21.15.98:0/1868202160 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:42.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f13fff640 1 --2- 172.21.15.98:0/1868202160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f1c14a040 0x7f0f1c14fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:42.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f213c7640 1 --2- 172.21.15.98:0/1868202160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c154210 0x7f0f1c1546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:42.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f20bc6640 1 -- 172.21.15.98:0/1868202160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c154210 msgr2=0x7f0f1c1546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f20bc6640 1 --2- 172.21.15.98:0/1868202160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c154210 0x7f0f1c1546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f20bc6640 1 -- 172.21.15.98:0/1868202160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f1c14a040 msgr2=0x7f0f1c14fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f20bc6640 1 --2- 172.21.15.98:0/1868202160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f1c14a040 0x7f0f1c14fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f20bc6640 1 -- 172.21.15.98:0/1868202160 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1c1f54f0 con 0x7f0f04004800 2024-09-10T20:59:42.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f20bc6640 1 --2- 172.21.15.98:0/1868202160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f04004800 0x7f0f1c14a640 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0f18004650 tx=0x7f0f1800bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:42.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f117fa640 1 -- 172.21.15.98:0/1868202160 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18013bb0 con 0x7f0f04004800 2024-09-10T20:59:42.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f117fa640 1 -- 172.21.15.98:0/1868202160 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f18017070 con 0x7f0f04004800 2024-09-10T20:59:42.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f22e2a640 1 -- 172.21.15.98:0/1868202160 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1c1f57d0 con 0x7f0f04004800 2024-09-10T20:59:42.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f117fa640 1 -- 172.21.15.98:0/1868202160 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f180105b0 con 0x7f0f04004800 2024-09-10T20:59:42.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.570+0000 7f0f22e2a640 1 -- 172.21.15.98:0/1868202160 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f1c14dcf0 con 0x7f0f04004800 2024-09-10T20:59:42.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.571+0000 7f0f22e2a640 1 -- 172.21.15.98:0/1868202160 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f1c14a420 con 0x7f0f04004800 2024-09-10T20:59:42.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.572+0000 7f0f117fa640 1 -- 172.21.15.98:0/1868202160 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0f18024030 con 0x7f0f04004800 2024-09-10T20:59:42.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.572+0000 7f0f117fa640 1 --2- 172.21.15.98:0/1868202160 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0f08077fa0 0x7f0f0807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:42.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.572+0000 7f0f117fa640 1 -- 172.21.15.98:0/1868202160 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f0f1809c670 con 0x7f0f04004800 2024-09-10T20:59:42.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.572+0000 7f0f13fff640 1 --2- 172.21.15.98:0/1868202160 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0f08077fa0 0x7f0f0807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:42.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.573+0000 7f0f13fff640 1 --2- 172.21.15.98:0/1868202160 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0f08077fa0 0x7f0f0807a460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c033000 tx=0x7f0f0c02d000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:42.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.577+0000 7f0f117fa640 1 -- 172.21.15.98:0/1868202160 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0f180653c0 con 0x7f0f04004800 2024-09-10T20:59:42.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-10T20:59:42.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.818+0000 7f0f22e2a640 1 -- 172.21.15.98:0/1868202160 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f0f1c15ed30 con 0x7f0f04004800 2024-09-10T20:59:42.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.823+0000 7f0f117fa640 1 -- 172.21.15.98:0/1868202160 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f0f18064cc0 con 0x7f0f04004800 2024-09-10T20:59:42.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.825+0000 7f0f22e2a640 1 -- 172.21.15.98:0/1868202160 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0f08077fa0 msgr2=0x7f0f0807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.825+0000 7f0f22e2a640 1 --2- 172.21.15.98:0/1868202160 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0f08077fa0 0x7f0f0807a460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c033000 tx=0x7f0f0c02d000 comp rx=0 tx=0).stop 2024-09-10T20:59:42.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.825+0000 7f0f22e2a640 1 -- 172.21.15.98:0/1868202160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f04004800 msgr2=0x7f0f1c14a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.825+0000 7f0f22e2a640 1 --2- 172.21.15.98:0/1868202160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f04004800 0x7f0f1c14a640 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0f18004650 tx=0x7f0f1800bac0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.826+0000 7f0f22e2a640 1 -- 172.21.15.98:0/1868202160 shutdown_connections 2024-09-10T20:59:42.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.826+0000 7f0f22e2a640 1 --2- 172.21.15.98:0/1868202160 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f0f08077fa0 0x7f0f0807a460 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.826+0000 7f0f22e2a640 1 --2- 172.21.15.98:0/1868202160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0f1c154210 0x7f0f1c1546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.826+0000 7f0f22e2a640 1 --2- 172.21.15.98:0/1868202160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0f1c14a040 0x7f0f1c14fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.826+0000 7f0f22e2a640 1 --2- 172.21.15.98:0/1868202160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f04004800 0x7f0f1c14a640 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.826+0000 7f0f22e2a640 1 -- 172.21.15.98:0/1868202160 >> 172.21.15.98:0/1868202160 conn(0x7f0f1c0d0c00 msgr2=0x7f0f1c0b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:42.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.826+0000 7f0f22e2a640 1 -- 172.21.15.98:0/1868202160 shutdown_connections 2024-09-10T20:59:42.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.826+0000 7f0f22e2a640 1 -- 172.21.15.98:0/1868202160 wait complete. 2024-09-10T20:59:42.841 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-10T20:59:42.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.939+0000 7fa9992b0640 1 -- 172.21.15.98:0/3959066984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9800053c0 msgr2=0x7fa98000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.939+0000 7fa9992b0640 1 --2- 172.21.15.98:0/3959066984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9800053c0 0x7fa98000b9c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c640 tx=0x7fa980005830 comp rx=0 tx=0).stop 2024-09-10T20:59:42.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.939+0000 7fa9992b0640 1 -- 172.21.15.98:0/3959066984 shutdown_connections 2024-09-10T20:59:42.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.939+0000 7fa9992b0640 1 --2- 172.21.15.98:0/3959066984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9800053c0 0x7fa98000b9c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.939+0000 7fa9992b0640 1 -- 172.21.15.98:0/3959066984 >> 172.21.15.98:0/3959066984 conn(0x7fa9940d3db0 msgr2=0x7fa9940d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:42.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.939+0000 7fa9992b0640 1 -- 172.21.15.98:0/3959066984 shutdown_connections 2024-09-10T20:59:42.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.939+0000 7fa9992b0640 1 -- 172.21.15.98:0/3959066984 wait complete. 2024-09-10T20:59:42.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.940+0000 7fa9992b0640 1 Processor -- start 2024-09-10T20:59:42.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.940+0000 7fa9992b0640 1 -- start start 2024-09-10T20:59:42.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9992b0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9800053c0 0x7fa9940a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9992b0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa99414fdd0 0x7fa9941f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9992b0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa99415a6c0 0x7fa99415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9992b0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa9940d2730 con 0x7fa99414fdd0 2024-09-10T20:59:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9992b0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa9940d28b0 con 0x7fa9800053c0 2024-09-10T20:59:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9992b0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa9941f8ae0 con 0x7fa99415a6c0 2024-09-10T20:59:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9927fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa99414fdd0 0x7fa9941f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9927fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa99414fdd0 0x7fa9941f85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45732/0 (socket says 172.21.15.98:45732) 2024-09-10T20:59:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9927fc640 1 -- 172.21.15.98:0/1654040220 learned_addr learned my addr 172.21.15.98:0/1654040220 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa992ffd640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9800053c0 0x7fa9940a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9937fe640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa99415a6c0 0x7fa99415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9927fc640 1 -- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa99415a6c0 msgr2=0x7fa99415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9927fc640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa99415a6c0 0x7fa99415ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9927fc640 1 -- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9800053c0 msgr2=0x7fa9940a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:42.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9927fc640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9800053c0 0x7fa9940a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:42.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9927fc640 1 -- 172.21.15.98:0/1654040220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa994159620 con 0x7fa99414fdd0 2024-09-10T20:59:42.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa992ffd640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9800053c0 0x7fa9940a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:42.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa9927fc640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa99414fdd0 0x7fa9941f85a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00f7d0 tx=0x7fa97c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:42.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.941+0000 7fa98b7fe640 1 -- 172.21.15.98:0/1654040220 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c00bcb0 con 0x7fa99414fdd0 2024-09-10T20:59:42.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.942+0000 7fa98b7fe640 1 -- 172.21.15.98:0/1654040220 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa97c004030 con 0x7fa99414fdd0 2024-09-10T20:59:42.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.942+0000 7fa9992b0640 1 -- 172.21.15.98:0/1654040220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa994159900 con 0x7fa99414fdd0 2024-09-10T20:59:42.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.942+0000 7fa98b7fe640 1 -- 172.21.15.98:0/1654040220 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c004480 con 0x7fa99414fdd0 2024-09-10T20:59:42.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.942+0000 7fa9992b0640 1 -- 172.21.15.98:0/1654040220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa994159ec0 con 0x7fa99414fdd0 2024-09-10T20:59:42.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.943+0000 7fa9992b0640 1 -- 172.21.15.98:0/1654040220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9940d1ab0 con 0x7fa99414fdd0 2024-09-10T20:59:42.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.943+0000 7fa98b7fe640 1 -- 172.21.15.98:0/1654040220 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa97c0035f0 con 0x7fa99414fdd0 2024-09-10T20:59:42.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.944+0000 7fa98b7fe640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa974077f70 0x7fa97407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:42.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.944+0000 7fa98b7fe640 1 -- 172.21.15.98:0/1654040220 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fa97c09a4f0 con 0x7fa99414fdd0 2024-09-10T20:59:42.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.944+0000 7fa992ffd640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa974077f70 0x7fa97407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:42.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.944+0000 7fa992ffd640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa974077f70 0x7fa97407a430 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa98c000c00 tx=0x7fa98c006b70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:42.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:42.948+0000 7fa98b7fe640 1 -- 172.21.15.98:0/1654040220 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa97c062b60 con 0x7fa99414fdd0 2024-09-10T20:59:43.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.188+0000 7fa9992b0640 1 -- 172.21.15.98:0/1654040220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fa9940d4310 con 0x7fa99414fdd0 2024-09-10T20:59:43.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.190+0000 7fa98b7fe640 1 -- 172.21.15.98:0/1654040220 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7fa97c067590 con 0x7fa99414fdd0 2024-09-10T20:59:43.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.193+0000 7fa9992b0640 1 -- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa974077f70 msgr2=0x7fa97407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.193+0000 7fa9992b0640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa974077f70 0x7fa97407a430 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa98c000c00 tx=0x7fa98c006b70 comp rx=0 tx=0).stop 2024-09-10T20:59:43.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.193+0000 7fa9992b0640 1 -- 172.21.15.98:0/1654040220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa99414fdd0 msgr2=0x7fa9941f85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.193+0000 7fa9992b0640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa99414fdd0 0x7fa9941f85a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00f7d0 tx=0x7fa97c00fc90 comp rx=0 tx=0).stop 2024-09-10T20:59:43.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.193+0000 7fa9992b0640 1 -- 172.21.15.98:0/1654040220 shutdown_connections 2024-09-10T20:59:43.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.193+0000 7fa9992b0640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa974077f70 0x7fa97407a430 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.193+0000 7fa9992b0640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa99415a6c0 0x7fa99415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.193+0000 7fa9992b0640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa99414fdd0 0x7fa9941f85a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.193+0000 7fa9992b0640 1 --2- 172.21.15.98:0/1654040220 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9800053c0 0x7fa9940a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.193+0000 7fa9992b0640 1 -- 172.21.15.98:0/1654040220 >> 172.21.15.98:0/1654040220 conn(0x7fa9940d3db0 msgr2=0x7fa99415caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:43.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.193+0000 7fa9992b0640 1 -- 172.21.15.98:0/1654040220 shutdown_connections 2024-09-10T20:59:43.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.193+0000 7fa9992b0640 1 -- 172.21.15.98:0/1654040220 wait complete. 2024-09-10T20:59:43.208 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-10T20:59:43.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.306+0000 7f1319bf3640 1 -- 172.21.15.98:0/3114480464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12fc004800 msgr2=0x7f12fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.306+0000 7f1319bf3640 1 --2- 172.21.15.98:0/3114480464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12fc004800 0x7f12fc004c80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1308001690 tx=0x7f12fc005830 comp rx=0 tx=0).stop 2024-09-10T20:59:43.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.306+0000 7f1319bf3640 1 -- 172.21.15.98:0/3114480464 shutdown_connections 2024-09-10T20:59:43.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.306+0000 7f1319bf3640 1 --2- 172.21.15.98:0/3114480464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12fc004800 0x7f12fc004c80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.306+0000 7f1319bf3640 1 -- 172.21.15.98:0/3114480464 >> 172.21.15.98:0/3114480464 conn(0x7f13140b1760 msgr2=0x7f13140b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:43.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.306+0000 7f1319bf3640 1 -- 172.21.15.98:0/3114480464 shutdown_connections 2024-09-10T20:59:43.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.306+0000 7f1319bf3640 1 -- 172.21.15.98:0/3114480464 wait complete. 2024-09-10T20:59:43.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.307+0000 7f1319bf3640 1 Processor -- start 2024-09-10T20:59:43.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.308+0000 7f1319bf3640 1 -- start start 2024-09-10T20:59:43.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.308+0000 7f1319bf3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12fc004800 0x7f13141f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:43.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.308+0000 7f1319bf3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13141451a0 0x7f1314152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:43.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.308+0000 7f1319bf3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1314158f00 0x7f1314159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:43.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.308+0000 7f1319bf3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f13140b4930 con 0x7f1314158f00 2024-09-10T20:59:43.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.308+0000 7f1319bf3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f13140b47b0 con 0x7f13141451a0 2024-09-10T20:59:43.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.308+0000 7f1319bf3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f13140b4ab0 con 0x7f12fc004800 2024-09-10T20:59:43.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.308+0000 7f1313fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1314158f00 0x7f1314159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:43.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.308+0000 7f1313fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1314158f00 0x7f1314159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45754/0 (socket says 172.21.15.98:45754) 2024-09-10T20:59:43.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.308+0000 7f1313fff640 1 -- 172.21.15.98:0/390024548 learned_addr learned my addr 172.21.15.98:0/390024548 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:43.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.309+0000 7f1312ffd640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13141451a0 0x7f1314152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:43.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.309+0000 7f13137fe640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12fc004800 0x7f13141f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:43.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.309+0000 7f1313fff640 1 -- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12fc004800 msgr2=0x7f13141f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.309+0000 7f1313fff640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12fc004800 0x7f13141f1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.309+0000 7f1313fff640 1 -- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13141451a0 msgr2=0x7f1314152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.309+0000 7f1313fff640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13141451a0 0x7f1314152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.309+0000 7f1313fff640 1 -- 172.21.15.98:0/390024548 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1314144d40 con 0x7f1314158f00 2024-09-10T20:59:43.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.309+0000 7f13137fe640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12fc004800 0x7f13141f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:43.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.309+0000 7f1313fff640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1314158f00 0x7f1314159380 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f131415b990 tx=0x7f130c004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:43.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.309+0000 7f12fbfff640 1 -- 172.21.15.98:0/390024548 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c010890 con 0x7f1314158f00 2024-09-10T20:59:43.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.309+0000 7f12fbfff640 1 -- 172.21.15.98:0/390024548 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f130c017070 con 0x7f1314158f00 2024-09-10T20:59:43.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.309+0000 7f12fbfff640 1 -- 172.21.15.98:0/390024548 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c010dc0 con 0x7f1314158f00 2024-09-10T20:59:43.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.309+0000 7f1319bf3640 1 -- 172.21.15.98:0/390024548 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1314158420 con 0x7f1314158f00 2024-09-10T20:59:43.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.309+0000 7f1319bf3640 1 -- 172.21.15.98:0/390024548 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13141589e0 con 0x7f1314158f00 2024-09-10T20:59:43.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.310+0000 7f1319bf3640 1 -- 172.21.15.98:0/390024548 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1314072740 con 0x7f1314158f00 2024-09-10T20:59:43.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.311+0000 7f12fbfff640 1 -- 172.21.15.98:0/390024548 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f130c01c460 con 0x7f1314158f00 2024-09-10T20:59:43.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.311+0000 7f12fbfff640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1300077f70 0x7f130007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:43.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.311+0000 7f12fbfff640 1 -- 172.21.15.98:0/390024548 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f130c09a420 con 0x7f1314158f00 2024-09-10T20:59:43.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.311+0000 7f13137fe640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1300077f70 0x7f130007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:43.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.312+0000 7f13137fe640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1300077f70 0x7f130007a430 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1304005fd0 tx=0x7f130400b3a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:43.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.315+0000 7f12fbfff640 1 -- 172.21.15.98:0/390024548 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1314073ef0 con 0x7f1314158f00 2024-09-10T20:59:43.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.555+0000 7f1319bf3640 1 -- 172.21.15.98:0/390024548 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f13140b4220 con 0x7f1314158f00 2024-09-10T20:59:43.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.557+0000 7f12fbfff640 1 -- 172.21.15.98:0/390024548 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f130c0630e0 con 0x7f1314158f00 2024-09-10T20:59:43.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.559+0000 7f1319bf3640 1 -- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1300077f70 msgr2=0x7f130007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.559+0000 7f1319bf3640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1300077f70 0x7f130007a430 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1304005fd0 tx=0x7f130400b3a0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.559+0000 7f1319bf3640 1 -- 172.21.15.98:0/390024548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1314158f00 msgr2=0x7f1314159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.559+0000 7f1319bf3640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1314158f00 0x7f1314159380 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f131415b990 tx=0x7f130c004f90 comp rx=0 tx=0).stop 2024-09-10T20:59:43.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.559+0000 7f1319bf3640 1 -- 172.21.15.98:0/390024548 shutdown_connections 2024-09-10T20:59:43.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.559+0000 7f1319bf3640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1300077f70 0x7f130007a430 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.559+0000 7f1319bf3640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1314158f00 0x7f1314159380 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.559+0000 7f1319bf3640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13141451a0 0x7f1314152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.559+0000 7f1319bf3640 1 --2- 172.21.15.98:0/390024548 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12fc004800 0x7f13141f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.559+0000 7f1319bf3640 1 -- 172.21.15.98:0/390024548 >> 172.21.15.98:0/390024548 conn(0x7f13140b1760 msgr2=0x7f131415c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:43.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.559+0000 7f1319bf3640 1 -- 172.21.15.98:0/390024548 shutdown_connections 2024-09-10T20:59:43.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.559+0000 7f1319bf3640 1 -- 172.21.15.98:0/390024548 wait complete. 2024-09-10T20:59:43.574 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T20:59:43.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.671+0000 7f7ccef6b640 1 -- 172.21.15.98:0/4245040414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cc806e2e0 msgr2=0x7f7cc815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.671+0000 7f7ccef6b640 1 --2- 172.21.15.98:0/4245040414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cc806e2e0 0x7f7cc815e530 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009830 tx=0x7f7cb802a510 comp rx=0 tx=0).stop 2024-09-10T20:59:43.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.671+0000 7f7ccef6b640 1 -- 172.21.15.98:0/4245040414 shutdown_connections 2024-09-10T20:59:43.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.671+0000 7f7ccef6b640 1 --2- 172.21.15.98:0/4245040414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cc80cf110 0x7f7cc80b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.671+0000 7f7ccef6b640 1 --2- 172.21.15.98:0/4245040414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cc806e2e0 0x7f7cc815e530 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.671+0000 7f7ccef6b640 1 -- 172.21.15.98:0/4245040414 >> 172.21.15.98:0/4245040414 conn(0x7f7cc80b1760 msgr2=0x7f7cc80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:43.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.671+0000 7f7ccef6b640 1 -- 172.21.15.98:0/4245040414 shutdown_connections 2024-09-10T20:59:43.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.671+0000 7f7ccef6b640 1 -- 172.21.15.98:0/4245040414 wait complete. 2024-09-10T20:59:43.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.672+0000 7f7ccef6b640 1 Processor -- start 2024-09-10T20:59:43.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.672+0000 7f7ccef6b640 1 -- start start 2024-09-10T20:59:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7ccef6b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cc806e2e0 0x7f7cc814f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7ccef6b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cc80cf110 0x7f7cc81f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7ccef6b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cc815e150 0x7f7cc81f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7ccef6b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc80cc570 con 0x7f7cc815e150 2024-09-10T20:59:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7ccef6b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc80729d0 con 0x7f7cc806e2e0 2024-09-10T20:59:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7ccef6b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc80cc6f0 con 0x7f7cc80cf110 2024-09-10T20:59:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7ccd508640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cc815e150 0x7f7cc81f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7ccd508640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cc815e150 0x7f7cc81f6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45768/0 (socket says 172.21.15.98:45768) 2024-09-10T20:59:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7ccd508640 1 -- 172.21.15.98:0/1421037360 learned_addr learned my addr 172.21.15.98:0/1421037360 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7cccd07640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cc806e2e0 0x7f7cc814f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7cbffff640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cc80cf110 0x7f7cc81f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:43.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7ccd508640 1 -- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cc80cf110 msgr2=0x7f7cc81f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7ccd508640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cc80cf110 0x7f7cc81f5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7ccd508640 1 -- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cc806e2e0 msgr2=0x7f7cc814f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7ccd508640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cc806e2e0 0x7f7cc814f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7ccd508640 1 -- 172.21.15.98:0/1421037360 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc8157ee0 con 0x7f7cc815e150 2024-09-10T20:59:43.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7cccd07640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cc806e2e0 0x7f7cc814f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:43.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.673+0000 7f7cbffff640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cc80cf110 0x7f7cc81f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:43.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.674+0000 7f7ccd508640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cc815e150 0x7f7cc81f6010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000efd0 tx=0x7f7cc000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:43.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.674+0000 7f7cbd7fa640 1 -- 172.21.15.98:0/1421037360 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0003b70 con 0x7f7cc815e150 2024-09-10T20:59:43.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.674+0000 7f7cbd7fa640 1 -- 172.21.15.98:0/1421037360 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cc0003e00 con 0x7f7cc815e150 2024-09-10T20:59:43.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.674+0000 7f7ccef6b640 1 -- 172.21.15.98:0/1421037360 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc8156880 con 0x7f7cc815e150 2024-09-10T20:59:43.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.674+0000 7f7cbd7fa640 1 -- 172.21.15.98:0/1421037360 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0004280 con 0x7f7cc815e150 2024-09-10T20:59:43.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.674+0000 7f7ccef6b640 1 -- 172.21.15.98:0/1421037360 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc8158480 con 0x7f7cc815e150 2024-09-10T20:59:43.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.675+0000 7f7ccef6b640 1 -- 172.21.15.98:0/1421037360 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc80ccba0 con 0x7f7cc815e150 2024-09-10T20:59:43.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.675+0000 7f7cbd7fa640 1 -- 172.21.15.98:0/1421037360 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7cc0004420 con 0x7f7cc815e150 2024-09-10T20:59:43.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.676+0000 7f7cbd7fa640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7cb4077f70 0x7f7cb407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:43.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.676+0000 7f7cbd7fa640 1 -- 172.21.15.98:0/1421037360 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f7cc009ab20 con 0x7f7cc815e150 2024-09-10T20:59:43.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.676+0000 7f7cccd07640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7cb4077f70 0x7f7cb407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:43.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.676+0000 7f7cccd07640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7cb4077f70 0x7f7cb407a430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4007fd0 tx=0x7f7cc4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:43.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.680+0000 7f7cbd7fa640 1 -- 172.21.15.98:0/1421037360 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7cc00637f0 con 0x7f7cc815e150 2024-09-10T20:59:43.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.856+0000 7f7ccef6b640 1 -- 172.21.15.98:0/1421037360 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7cc814fb30 con 0x7f7cc815e150 2024-09-10T20:59:43.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.857+0000 7f7cbd7fa640 1 -- 172.21.15.98:0/1421037360 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) ==== 84+0+65 (secure 0 0 0) 0x7f7cc00630f0 con 0x7f7cc815e150 2024-09-10T20:59:43.861 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:43.861 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-10T20:59:43.861 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-10T20:59:43.861 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-09-10T20:59:43.861 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-09-10T20:59:43.862 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-10T20:59:43.862 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-10T20:59:43.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.859+0000 7f7ccef6b640 1 -- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7cb4077f70 msgr2=0x7f7cb407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.859+0000 7f7ccef6b640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7cb4077f70 0x7f7cb407a430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4007fd0 tx=0x7f7cc4009040 comp rx=0 tx=0).stop 2024-09-10T20:59:43.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.859+0000 7f7ccef6b640 1 -- 172.21.15.98:0/1421037360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cc815e150 msgr2=0x7f7cc81f6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.859+0000 7f7ccef6b640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cc815e150 0x7f7cc81f6010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000efd0 tx=0x7f7cc000c6a0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.859+0000 7f7ccef6b640 1 -- 172.21.15.98:0/1421037360 shutdown_connections 2024-09-10T20:59:43.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.859+0000 7f7ccef6b640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7cb4077f70 0x7f7cb407a430 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.859+0000 7f7ccef6b640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cc815e150 0x7f7cc81f6010 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.859+0000 7f7ccef6b640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cc80cf110 0x7f7cc81f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.859+0000 7f7ccef6b640 1 --2- 172.21.15.98:0/1421037360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7cc806e2e0 0x7f7cc814f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.859+0000 7f7ccef6b640 1 -- 172.21.15.98:0/1421037360 >> 172.21.15.98:0/1421037360 conn(0x7f7cc80b1760 msgr2=0x7f7cc80cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:43.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.859+0000 7f7ccef6b640 1 -- 172.21.15.98:0/1421037360 shutdown_connections 2024-09-10T20:59:43.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.859+0000 7f7ccef6b640 1 -- 172.21.15.98:0/1421037360 wait complete. 2024-09-10T20:59:43.874 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T20:59:43.874 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T20:59:43.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.975+0000 7ff888ba0640 1 -- 172.21.15.98:0/3245903815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff86c004050 msgr2=0x7ff86c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.975+0000 7ff888ba0640 1 --2- 172.21.15.98:0/3245903815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff86c004050 0x7ff86c004800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff878001630 tx=0x7ff86c005830 comp rx=0 tx=0).stop 2024-09-10T20:59:43.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.975+0000 7ff882d9d640 1 -- 172.21.15.98:0/3245903815 reap_dead start 2024-09-10T20:59:43.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.975+0000 7ff888ba0640 1 -- 172.21.15.98:0/3245903815 shutdown_connections 2024-09-10T20:59:43.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.975+0000 7ff888ba0640 1 -- 172.21.15.98:0/3245903815 >> 172.21.15.98:0/3245903815 conn(0x7ff884074150 msgr2=0x7ff884074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:43.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.975+0000 7ff888ba0640 1 -- 172.21.15.98:0/3245903815 shutdown_connections 2024-09-10T20:59:43.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.975+0000 7ff888ba0640 1 -- 172.21.15.98:0/3245903815 wait complete. 2024-09-10T20:59:43.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.976+0000 7ff888ba0640 1 Processor -- start 2024-09-10T20:59:43.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.976+0000 7ff888ba0640 1 -- start start 2024-09-10T20:59:43.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.977+0000 7ff888ba0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff86c004050 0x7ff8841f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:43.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.977+0000 7ff888ba0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8840ca490 0x7ff8840d0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:43.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.977+0000 7ff888ba0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8840cf1b0 0x7ff8840d0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:43.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.977+0000 7ff888ba0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff88406cf00 con 0x7ff8840cf1b0 2024-09-10T20:59:43.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.977+0000 7ff888ba0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff88406cd80 con 0x7ff8840ca490 2024-09-10T20:59:43.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.977+0000 7ff888ba0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff8840b1740 con 0x7ff86c004050 2024-09-10T20:59:43.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.977+0000 7ff882d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8840cf1b0 0x7ff8840d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:43.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.977+0000 7ff882d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8840cf1b0 0x7ff8840d0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45792/0 (socket says 172.21.15.98:45792) 2024-09-10T20:59:43.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.977+0000 7ff882d9d640 1 -- 172.21.15.98:0/839900844 learned_addr learned my addr 172.21.15.98:0/839900844 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:43.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.977+0000 7ff882d9d640 1 -- 172.21.15.98:0/839900844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff86c004050 msgr2=0x7ff8841f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.977+0000 7ff882d9d640 1 --2- 172.21.15.98:0/839900844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff86c004050 0x7ff8841f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.977+0000 7ff882d9d640 1 -- 172.21.15.98:0/839900844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8840ca490 msgr2=0x7ff8840d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:43.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.977+0000 7ff882d9d640 1 --2- 172.21.15.98:0/839900844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8840ca490 0x7ff8840d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:43.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.977+0000 7ff882d9d640 1 -- 172.21.15.98:0/839900844 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8840afce0 con 0x7ff8840cf1b0 2024-09-10T20:59:43.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.978+0000 7ff882d9d640 1 --2- 172.21.15.98:0/839900844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8840cf1b0 0x7ff8840d0860 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff884072d10 tx=0x7ff87c010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:43.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.978+0000 7ff872ffd640 1 -- 172.21.15.98:0/839900844 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c003e40 con 0x7ff8840cf1b0 2024-09-10T20:59:43.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.978+0000 7ff872ffd640 1 -- 172.21.15.98:0/839900844 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff87c015070 con 0x7ff8840cf1b0 2024-09-10T20:59:43.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.978+0000 7ff888ba0640 1 -- 172.21.15.98:0/839900844 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8840cbe80 con 0x7ff8840cf1b0 2024-09-10T20:59:43.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.978+0000 7ff872ffd640 1 -- 172.21.15.98:0/839900844 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00ec90 con 0x7ff8840cf1b0 2024-09-10T20:59:43.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.978+0000 7ff888ba0640 1 -- 172.21.15.98:0/839900844 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88415ba20 con 0x7ff8840cf1b0 2024-09-10T20:59:43.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.979+0000 7ff888ba0640 1 -- 172.21.15.98:0/839900844 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8840ca970 con 0x7ff8840cf1b0 2024-09-10T20:59:43.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.979+0000 7ff872ffd640 1 -- 172.21.15.98:0/839900844 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff87c024060 con 0x7ff8840cf1b0 2024-09-10T20:59:43.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.980+0000 7ff872ffd640 1 --2- 172.21.15.98:0/839900844 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff864077fa0 0x7ff86407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:43.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.980+0000 7ff872ffd640 1 -- 172.21.15.98:0/839900844 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7ff87c099c70 con 0x7ff8840cf1b0 2024-09-10T20:59:43.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.980+0000 7ff88259c640 1 --2- 172.21.15.98:0/839900844 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff864077fa0 0x7ff86407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:43.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.980+0000 7ff88259c640 1 --2- 172.21.15.98:0/839900844 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff864077fa0 0x7ff86407a460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff874000fb0 tx=0x7ff874001e10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:43.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:43.984+0000 7ff872ffd640 1 -- 172.21.15.98:0/839900844 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff87c0629c0 con 0x7ff8840cf1b0 2024-09-10T20:59:44.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:44.158+0000 7ff888ba0640 1 -- 172.21.15.98:0/839900844 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff8840cc6e0 con 0x7ff8840cf1b0 2024-09-10T20:59:44.335 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T20:59:44.337 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T20:59:44.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:44.990+0000 7ff872ffd640 1 -- 172.21.15.98:0/839900844 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) ==== 138+0+0 (secure 0 0 0) 0x7ff87c0622c0 con 0x7ff8840cf1b0 2024-09-10T20:59:44.995 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-09-10T20:59:44.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:44.994+0000 7ff888ba0640 1 -- 172.21.15.98:0/839900844 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff864077fa0 msgr2=0x7ff86407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:44.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:44.994+0000 7ff888ba0640 1 --2- 172.21.15.98:0/839900844 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff864077fa0 0x7ff86407a460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff874000fb0 tx=0x7ff874001e10 comp rx=0 tx=0).stop 2024-09-10T20:59:44.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:44.994+0000 7ff888ba0640 1 -- 172.21.15.98:0/839900844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8840cf1b0 msgr2=0x7ff8840d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:44.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:44.994+0000 7ff888ba0640 1 --2- 172.21.15.98:0/839900844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8840cf1b0 0x7ff8840d0860 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff884072d10 tx=0x7ff87c010e50 comp rx=0 tx=0).stop 2024-09-10T20:59:44.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:44.995+0000 7ff888ba0640 1 -- 172.21.15.98:0/839900844 shutdown_connections 2024-09-10T20:59:44.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:44.995+0000 7ff888ba0640 1 --2- 172.21.15.98:0/839900844 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff864077fa0 0x7ff86407a460 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:44.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:44.995+0000 7ff888ba0640 1 --2- 172.21.15.98:0/839900844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8840cf1b0 0x7ff8840d0860 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:44.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:44.995+0000 7ff888ba0640 1 --2- 172.21.15.98:0/839900844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8840ca490 0x7ff8840d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:44.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:44.995+0000 7ff888ba0640 1 --2- 172.21.15.98:0/839900844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff86c004050 0x7ff8841f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:44.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:44.995+0000 7ff888ba0640 1 -- 172.21.15.98:0/839900844 >> 172.21.15.98:0/839900844 conn(0x7ff884074150 msgr2=0x7ff8840af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:45.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:44.996+0000 7ff888ba0640 1 -- 172.21.15.98:0/839900844 shutdown_connections 2024-09-10T20:59:45.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:44.996+0000 7ff888ba0640 1 -- 172.21.15.98:0/839900844 wait complete. 2024-09-10T20:59:45.015 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T20:59:45.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.125+0000 7fb557ab8640 1 -- 172.21.15.98:0/1176654878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb55006e0a0 msgr2=0x7fb5500ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:45.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.125+0000 7fb557ab8640 1 --2- 172.21.15.98:0/1176654878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb55006e0a0 0x7fb5500ca970 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb540005fd0 tx=0x7fb54002a4b0 comp rx=0 tx=0).stop 2024-09-10T20:59:45.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.125+0000 7fb557ab8640 1 -- 172.21.15.98:0/1176654878 shutdown_connections 2024-09-10T20:59:45.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.125+0000 7fb557ab8640 1 --2- 172.21.15.98:0/1176654878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb55006e0a0 0x7fb5500ca970 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:45.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.125+0000 7fb557ab8640 1 --2- 172.21.15.98:0/1176654878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb55006dcc0 0x7fb5500b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:45.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.125+0000 7fb557ab8640 1 -- 172.21.15.98:0/1176654878 >> 172.21.15.98:0/1176654878 conn(0x7fb550074150 msgr2=0x7fb550074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:45.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.125+0000 7fb557ab8640 1 -- 172.21.15.98:0/1176654878 shutdown_connections 2024-09-10T20:59:45.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.125+0000 7fb557ab8640 1 -- 172.21.15.98:0/1176654878 wait complete. 2024-09-10T20:59:45.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.126+0000 7fb557ab8640 1 Processor -- start 2024-09-10T20:59:45.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.126+0000 7fb557ab8640 1 -- start start 2024-09-10T20:59:45.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb557ab8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb55006cb70 0x7fb5501f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:45.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb557ab8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb55006dcc0 0x7fb55015b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:45.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb557ab8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb55006e0a0 0x7fb5500b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:45.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb557ab8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb5500d0320 con 0x7fb55006cb70 2024-09-10T20:59:45.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb557ab8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb550069490 con 0x7fb55006dcc0 2024-09-10T20:59:45.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb557ab8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb5500d04a0 con 0x7fb55006e0a0 2024-09-10T20:59:45.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb555854640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb55006cb70 0x7fb5501f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:45.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb555854640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb55006cb70 0x7fb5501f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45814/0 (socket says 172.21.15.98:45814) 2024-09-10T20:59:45.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb555854640 1 -- 172.21.15.98:0/3595056293 learned_addr learned my addr 172.21.15.98:0/3595056293 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:45.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb555053640 1 --2- 172.21.15.98:0/3595056293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb55006dcc0 0x7fb55015b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:45.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb556055640 1 --2- 172.21.15.98:0/3595056293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb55006e0a0 0x7fb5500b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:45.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb555854640 1 -- 172.21.15.98:0/3595056293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb55006e0a0 msgr2=0x7fb5500b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:45.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb555854640 1 --2- 172.21.15.98:0/3595056293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb55006e0a0 0x7fb5500b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:45.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb555854640 1 -- 172.21.15.98:0/3595056293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb55006dcc0 msgr2=0x7fb55015b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:45.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb555854640 1 --2- 172.21.15.98:0/3595056293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb55006dcc0 0x7fb55015b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:45.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.127+0000 7fb555854640 1 -- 172.21.15.98:0/3595056293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55015b6f0 con 0x7fb55006cb70 2024-09-10T20:59:45.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.128+0000 7fb555854640 1 --2- 172.21.15.98:0/3595056293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb55006cb70 0x7fb5501f63b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00d950 tx=0x7fb54c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:45.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.128+0000 7fb5467fc640 1 -- 172.21.15.98:0/3595056293 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c017030 con 0x7fb55006cb70 2024-09-10T20:59:45.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.128+0000 7fb5467fc640 1 -- 172.21.15.98:0/3595056293 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb54c009b20 con 0x7fb55006cb70 2024-09-10T20:59:45.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.128+0000 7fb557ab8640 1 -- 172.21.15.98:0/3595056293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5500af3b0 con 0x7fb55006cb70 2024-09-10T20:59:45.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.128+0000 7fb5467fc640 1 -- 172.21.15.98:0/3595056293 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c004020 con 0x7fb55006cb70 2024-09-10T20:59:45.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.128+0000 7fb557ab8640 1 -- 172.21.15.98:0/3595056293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5500c4050 con 0x7fb55006cb70 2024-09-10T20:59:45.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.129+0000 7fb557ab8640 1 -- 172.21.15.98:0/3595056293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5500b1740 con 0x7fb55006cb70 2024-09-10T20:59:45.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.129+0000 7fb5467fc640 1 -- 172.21.15.98:0/3595056293 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb54c0041c0 con 0x7fb55006cb70 2024-09-10T20:59:45.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.130+0000 7fb5467fc640 1 --2- 172.21.15.98:0/3595056293 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fb534077f70 0x7fb53407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:45.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.130+0000 7fb5467fc640 1 -- 172.21.15.98:0/3595056293 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5619+0+0 (secure 0 0 0) 0x7fb54c099dc0 con 0x7fb55006cb70 2024-09-10T20:59:45.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.130+0000 7fb555053640 1 --2- 172.21.15.98:0/3595056293 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fb534077f70 0x7fb53407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:45.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.131+0000 7fb555053640 1 --2- 172.21.15.98:0/3595056293 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fb534077f70 0x7fb53407a430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb540033000 tx=0x7fb540004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:45.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.134+0000 7fb5467fc640 1 -- 172.21.15.98:0/3595056293 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb54c0622c0 con 0x7fb55006cb70 2024-09-10T20:59:45.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.309+0000 7fb557ab8640 1 -- 172.21.15.98:0/3595056293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb550157200 con 0x7fb55006cb70 2024-09-10T20:59:45.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.992+0000 7fb5467fc640 1 -- 172.21.15.98:0/3595056293 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) ==== 170+0+0 (secure 0 0 0) 0x7fb54c066cf0 con 0x7fb55006cb70 2024-09-10T20:59:45.997 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-09-10T20:59:45.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.995+0000 7fb557ab8640 1 -- 172.21.15.98:0/3595056293 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fb534077f70 msgr2=0x7fb53407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:45.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.995+0000 7fb557ab8640 1 --2- 172.21.15.98:0/3595056293 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fb534077f70 0x7fb53407a430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb540033000 tx=0x7fb540004730 comp rx=0 tx=0).stop 2024-09-10T20:59:45.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.995+0000 7fb557ab8640 1 -- 172.21.15.98:0/3595056293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb55006cb70 msgr2=0x7fb5501f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:45.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.995+0000 7fb557ab8640 1 --2- 172.21.15.98:0/3595056293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb55006cb70 0x7fb5501f63b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00d950 tx=0x7fb54c00de10 comp rx=0 tx=0).stop 2024-09-10T20:59:45.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.995+0000 7fb557ab8640 1 -- 172.21.15.98:0/3595056293 shutdown_connections 2024-09-10T20:59:45.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.995+0000 7fb557ab8640 1 --2- 172.21.15.98:0/3595056293 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fb534077f70 0x7fb53407a430 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:45.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.995+0000 7fb557ab8640 1 --2- 172.21.15.98:0/3595056293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb55006e0a0 0x7fb5500b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:45.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.995+0000 7fb557ab8640 1 --2- 172.21.15.98:0/3595056293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb55006dcc0 0x7fb55015b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:46.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.995+0000 7fb557ab8640 1 --2- 172.21.15.98:0/3595056293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb55006cb70 0x7fb5501f63b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:46.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.995+0000 7fb557ab8640 1 -- 172.21.15.98:0/3595056293 >> 172.21.15.98:0/3595056293 conn(0x7fb550074150 msgr2=0x7fb5500ce850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:46.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.995+0000 7fb557ab8640 1 -- 172.21.15.98:0/3595056293 shutdown_connections 2024-09-10T20:59:46.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:45.995+0000 7fb557ab8640 1 -- 172.21.15.98:0/3595056293 wait complete. 2024-09-10T20:59:46.020 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T20:59:46.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.122+0000 7f37d2dc2640 1 -- 172.21.15.98:0/1736585381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37b4004b10 msgr2=0x7f37b4004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:46.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.122+0000 7f37d2dc2640 1 --2- 172.21.15.98:0/1736585381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37b4004b10 0x7f37b4004f30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00c4a0 tx=0x7f37b4006940 comp rx=0 tx=0).stop 2024-09-10T20:59:46.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.122+0000 7f37d135f640 1 -- 172.21.15.98:0/1736585381 reap_dead start 2024-09-10T20:59:46.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.122+0000 7f37d2dc2640 1 -- 172.21.15.98:0/1736585381 shutdown_connections 2024-09-10T20:59:46.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.122+0000 7f37d2dc2640 1 -- 172.21.15.98:0/1736585381 >> 172.21.15.98:0/1736585381 conn(0x7f37cc0b1760 msgr2=0x7f37cc0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:46.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.122+0000 7f37d2dc2640 1 -- 172.21.15.98:0/1736585381 shutdown_connections 2024-09-10T20:59:46.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.122+0000 7f37d2dc2640 1 -- 172.21.15.98:0/1736585381 wait complete. 2024-09-10T20:59:46.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.123+0000 7f37d2dc2640 1 Processor -- start 2024-09-10T20:59:46.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.123+0000 7f37d2dc2640 1 -- start start 2024-09-10T20:59:46.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.124+0000 7f37d2dc2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37b4004b10 0x7f37cc144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:46.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.124+0000 7f37d2dc2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc1458e0 0x7f37cc0aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:46.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.124+0000 7f37d2dc2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37cc15c690 0x7f37cc15cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:46.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.124+0000 7f37d2dc2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc0725d0 con 0x7f37cc1458e0 2024-09-10T20:59:46.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.124+0000 7f37d2dc2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc072750 con 0x7f37cc15c690 2024-09-10T20:59:46.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.124+0000 7f37d2dc2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f37cc0728d0 con 0x7f37b4004b10 2024-09-10T20:59:46.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.124+0000 7f37cbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc1458e0 0x7f37cc0aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:46.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.124+0000 7f37cbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc1458e0 0x7f37cc0aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45830/0 (socket says 172.21.15.98:45830) 2024-09-10T20:59:46.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.124+0000 7f37cbfff640 1 -- 172.21.15.98:0/3905153690 learned_addr learned my addr 172.21.15.98:0/3905153690 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:46.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.125+0000 7f37d135f640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37cc15c690 0x7f37cc15cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:46.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.125+0000 7f37d0b5e640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37b4004b10 0x7f37cc144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:46.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.125+0000 7f37cbfff640 1 -- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37b4004b10 msgr2=0x7f37cc144e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:46.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.125+0000 7f37cbfff640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37b4004b10 0x7f37cc144e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:46.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.125+0000 7f37cbfff640 1 -- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37cc15c690 msgr2=0x7f37cc15cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:46.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.125+0000 7f37cbfff640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37cc15c690 0x7f37cc15cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:46.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.125+0000 7f37cbfff640 1 -- 172.21.15.98:0/3905153690 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc1423f0 con 0x7f37cc1458e0 2024-09-10T20:59:46.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.125+0000 7f37d0b5e640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37b4004b10 0x7f37cc144e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:46.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.125+0000 7f37cbfff640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc1458e0 0x7f37cc0aedc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f37c0011a50 tx=0x7f37c0011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:46.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.125+0000 7f37c97fa640 1 -- 172.21.15.98:0/3905153690 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c001f070 con 0x7f37cc1458e0 2024-09-10T20:59:46.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.125+0000 7f37c97fa640 1 -- 172.21.15.98:0/3905153690 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37c000fe80 con 0x7f37cc1458e0 2024-09-10T20:59:46.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.125+0000 7f37c97fa640 1 -- 172.21.15.98:0/3905153690 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0012690 con 0x7f37cc1458e0 2024-09-10T20:59:46.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.125+0000 7f37d2dc2640 1 -- 172.21.15.98:0/3905153690 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc1453d0 con 0x7f37cc1458e0 2024-09-10T20:59:46.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.125+0000 7f37d2dc2640 1 -- 172.21.15.98:0/3905153690 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc142950 con 0x7f37cc1458e0 2024-09-10T20:59:46.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.126+0000 7f37d2dc2640 1 -- 172.21.15.98:0/3905153690 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37cc0b47b0 con 0x7f37cc1458e0 2024-09-10T20:59:46.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.127+0000 7f37c97fa640 1 -- 172.21.15.98:0/3905153690 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f37c0012830 con 0x7f37cc1458e0 2024-09-10T20:59:46.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.127+0000 7f37c97fa640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f37b8077f70 0x7f37b807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:46.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.127+0000 7f37c97fa640 1 -- 172.21.15.98:0/3905153690 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6001+0+0 (secure 0 0 0) 0x7f37c009abe0 con 0x7f37cc1458e0 2024-09-10T20:59:46.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.128+0000 7f37d0b5e640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f37b8077f70 0x7f37b807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:46.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.128+0000 7f37d0b5e640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f37b8077f70 0x7f37b807a430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f37bc033000 tx=0x7f37cc15b990 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:46.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.132+0000 7f37c97fa640 1 -- 172.21.15.98:0/3905153690 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f37cc1453d0 con 0x7f37cc1458e0 2024-09-10T20:59:46.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:46.342+0000 7f37d2dc2640 1 -- 172.21.15.98:0/3905153690 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f37cc06f1a0 con 0x7f37cc1458e0 2024-09-10T20:59:47.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.017+0000 7f37c97fa640 1 -- 172.21.15.98:0/3905153690 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f37cc1423f0 con 0x7f37cc1458e0 2024-09-10T20:59:47.022 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 5 and data pool 6 2024-09-10T20:59:47.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.025+0000 7f37d2dc2640 1 -- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f37b8077f70 msgr2=0x7f37b807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.025+0000 7f37d2dc2640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f37b8077f70 0x7f37b807a430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f37bc033000 tx=0x7f37cc15b990 comp rx=0 tx=0).stop 2024-09-10T20:59:47.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.025+0000 7f37d2dc2640 1 -- 172.21.15.98:0/3905153690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc1458e0 msgr2=0x7f37cc0aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.025+0000 7f37d2dc2640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc1458e0 0x7f37cc0aedc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f37c0011a50 tx=0x7f37c0011f10 comp rx=0 tx=0).stop 2024-09-10T20:59:47.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.026+0000 7f37d2dc2640 1 -- 172.21.15.98:0/3905153690 shutdown_connections 2024-09-10T20:59:47.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.026+0000 7f37d2dc2640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f37b8077f70 0x7f37b807a430 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.026+0000 7f37d2dc2640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37cc15c690 0x7f37cc15cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.026+0000 7f37d2dc2640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37cc1458e0 0x7f37cc0aedc0 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f37c0011a50 tx=0x7f37c0011f10 comp rx=0 tx=0).stop 2024-09-10T20:59:47.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.026+0000 7f37d2dc2640 1 --2- 172.21.15.98:0/3905153690 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37b4004b10 0x7f37cc144e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.026+0000 7f37d2dc2640 1 -- 172.21.15.98:0/3905153690 >> 172.21.15.98:0/3905153690 conn(0x7f37cc0b1760 msgr2=0x7f37cc0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:47.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.026+0000 7f37d2dc2640 1 -- 172.21.15.98:0/3905153690 shutdown_connections 2024-09-10T20:59:47.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.027+0000 7f37d2dc2640 1 -- 172.21.15.98:0/3905153690 wait complete. 2024-09-10T20:59:47.050 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:59:47.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.148+0000 7fd8848ed640 1 -- 172.21.15.98:0/4109147408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd880072680 msgr2=0x7fd8800ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.148+0000 7fd8848ed640 1 --2- 172.21.15.98:0/4109147408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd880072680 0x7fd8800ceb00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd88006de10 tx=0x7fd87402a4d0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.148+0000 7fd8848ed640 1 -- 172.21.15.98:0/4109147408 shutdown_connections 2024-09-10T20:59:47.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.148+0000 7fd8848ed640 1 --2- 172.21.15.98:0/4109147408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd880072ad0 0x7fd8800cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.148+0000 7fd8848ed640 1 --2- 172.21.15.98:0/4109147408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd880072680 0x7fd8800ceb00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.148+0000 7fd8848ed640 1 --2- 172.21.15.98:0/4109147408 >> v2:172.21.15.190:3300/0 conn(0x7fd880144780 0x7fd8800b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.148+0000 7fd8848ed640 1 -- 172.21.15.98:0/4109147408 >> 172.21.15.98:0/4109147408 conn(0x7fd8800b1760 msgr2=0x7fd8800b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:47.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.148+0000 7fd8848ed640 1 -- 172.21.15.98:0/4109147408 shutdown_connections 2024-09-10T20:59:47.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.148+0000 7fd8848ed640 1 -- 172.21.15.98:0/4109147408 wait complete. 2024-09-10T20:59:47.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.149+0000 7fd8848ed640 1 Processor -- start 2024-09-10T20:59:47.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.149+0000 7fd8848ed640 1 -- start start 2024-09-10T20:59:47.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd8848ed640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd880072680 0x7fd8800cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd8848ed640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd880072ad0 0x7fd880142490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd8848ed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd880144780 0x7fd8801429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd8848ed640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd88015c8e0 con 0x7fd880072ad0 2024-09-10T20:59:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd8848ed640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd88015c760 con 0x7fd880144780 2024-09-10T20:59:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd8848ed640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd88015ca60 con 0x7fd880072680 2024-09-10T20:59:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd87dd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd880072ad0 0x7fd880142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd87dd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd880072ad0 0x7fd880142490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45860/0 (socket says 172.21.15.98:45860) 2024-09-10T20:59:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd87dd9b640 1 -- 172.21.15.98:0/3939871918 learned_addr learned my addr 172.21.15.98:0/3939871918 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd87e59c640 1 --2- 172.21.15.98:0/3939871918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd880072680 0x7fd8800cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd87ed9d640 1 --2- 172.21.15.98:0/3939871918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd880144780 0x7fd8801429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd87dd9b640 1 -- 172.21.15.98:0/3939871918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd880072680 msgr2=0x7fd8800cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd87dd9b640 1 --2- 172.21.15.98:0/3939871918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd880072680 0x7fd8800cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd87dd9b640 1 -- 172.21.15.98:0/3939871918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd880144780 msgr2=0x7fd8801429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd87dd9b640 1 --2- 172.21.15.98:0/3939871918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd880144780 0x7fd8801429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.150+0000 7fd87dd9b640 1 -- 172.21.15.98:0/3939871918 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8801f2a10 con 0x7fd880072ad0 2024-09-10T20:59:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.151+0000 7fd87dd9b640 1 --2- 172.21.15.98:0/3939871918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd880072ad0 0x7fd880142490 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd874000c00 tx=0x7fd8740039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.151+0000 7fd86effd640 1 -- 172.21.15.98:0/3939871918 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87403f070 con 0x7fd880072ad0 2024-09-10T20:59:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.151+0000 7fd86effd640 1 -- 172.21.15.98:0/3939871918 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd874004890 con 0x7fd880072ad0 2024-09-10T20:59:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.151+0000 7fd8848ed640 1 -- 172.21.15.98:0/3939871918 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8801f2cf0 con 0x7fd880072ad0 2024-09-10T20:59:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.151+0000 7fd86effd640 1 -- 172.21.15.98:0/3939871918 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874035030 con 0x7fd880072ad0 2024-09-10T20:59:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.151+0000 7fd8848ed640 1 -- 172.21.15.98:0/3939871918 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8800cc6c0 con 0x7fd880072ad0 2024-09-10T20:59:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.152+0000 7fd8848ed640 1 -- 172.21.15.98:0/3939871918 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd88015cf90 con 0x7fd880072ad0 2024-09-10T20:59:47.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.152+0000 7fd86effd640 1 -- 172.21.15.98:0/3939871918 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd874004ca0 con 0x7fd880072ad0 2024-09-10T20:59:47.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.153+0000 7fd86effd640 1 --2- 172.21.15.98:0/3939871918 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd860077f70 0x7fd86007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:47.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.153+0000 7fd86effd640 1 -- 172.21.15.98:0/3939871918 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fd8740bb060 con 0x7fd880072ad0 2024-09-10T20:59:47.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.153+0000 7fd87e59c640 1 --2- 172.21.15.98:0/3939871918 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd860077f70 0x7fd86007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:47.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.154+0000 7fd87e59c640 1 --2- 172.21.15.98:0/3939871918 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd860077f70 0x7fd86007a430 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd8780098e0 tx=0x7fd878009060 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:47.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.157+0000 7fd86effd640 1 -- 172.21.15.98:0/3939871918 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd874083410 con 0x7fd880072ad0 2024-09-10T20:59:47.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.330+0000 7fd8848ed640 1 -- 172.21.15.98:0/3939871918 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8800cf810 con 0x7fd880072ad0 2024-09-10T20:59:47.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.331+0000 7fd86effd640 1 -- 172.21.15.98:0/3939871918 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17027 (secure 0 0 0) 0x7fd87408c180 con 0x7fd880072ad0 2024-09-10T20:59:47.335 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:47.336 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":39,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:59:46.344165+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:59:44.162419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T20:59:45.314201+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:59:47.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.333+0000 7fd8848ed640 1 -- 172.21.15.98:0/3939871918 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd860077f70 msgr2=0x7fd86007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.333+0000 7fd8848ed640 1 --2- 172.21.15.98:0/3939871918 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd860077f70 0x7fd86007a430 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd8780098e0 tx=0x7fd878009060 comp rx=0 tx=0).stop 2024-09-10T20:59:47.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.333+0000 7fd8848ed640 1 -- 172.21.15.98:0/3939871918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd880072ad0 msgr2=0x7fd880142490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.333+0000 7fd8848ed640 1 --2- 172.21.15.98:0/3939871918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd880072ad0 0x7fd880142490 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd874000c00 tx=0x7fd8740039e0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.333+0000 7fd8848ed640 1 -- 172.21.15.98:0/3939871918 shutdown_connections 2024-09-10T20:59:47.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.333+0000 7fd8848ed640 1 --2- 172.21.15.98:0/3939871918 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd860077f70 0x7fd86007a430 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.333+0000 7fd8848ed640 1 --2- 172.21.15.98:0/3939871918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd880144780 0x7fd8801429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.333+0000 7fd8848ed640 1 --2- 172.21.15.98:0/3939871918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd880072ad0 0x7fd880142490 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.333+0000 7fd8848ed640 1 --2- 172.21.15.98:0/3939871918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd880072680 0x7fd8800cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.333+0000 7fd8848ed640 1 -- 172.21.15.98:0/3939871918 >> 172.21.15.98:0/3939871918 conn(0x7fd8800b1760 msgr2=0x7fd8800cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:47.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.333+0000 7fd8848ed640 1 -- 172.21.15.98:0/3939871918 shutdown_connections 2024-09-10T20:59:47.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.333+0000 7fd8848ed640 1 -- 172.21.15.98:0/3939871918 wait complete. 2024-09-10T20:59:47.349 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:59:47.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.445+0000 7fad302bd640 1 -- 172.21.15.98:0/2429266110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad14003a50 msgr2=0x7fad14003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.445+0000 7fad302bd640 1 --2- 172.21.15.98:0/2429266110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad14003a50 0x7fad14003e70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fad2400bbb0 tx=0x7fad14005830 comp rx=0 tx=0).stop 2024-09-10T20:59:47.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.445+0000 7fad302bd640 1 -- 172.21.15.98:0/2429266110 shutdown_connections 2024-09-10T20:59:47.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.445+0000 7fad302bd640 1 --2- 172.21.15.98:0/2429266110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad14003a50 0x7fad14003e70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.445+0000 7fad302bd640 1 -- 172.21.15.98:0/2429266110 >> 172.21.15.98:0/2429266110 conn(0x7fad280d3f70 msgr2=0x7fad280d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:47.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.445+0000 7fad302bd640 1 -- 172.21.15.98:0/2429266110 shutdown_connections 2024-09-10T20:59:47.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.445+0000 7fad302bd640 1 -- 172.21.15.98:0/2429266110 wait complete. 2024-09-10T20:59:47.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.446+0000 7fad302bd640 1 Processor -- start 2024-09-10T20:59:47.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.446+0000 7fad302bd640 1 -- start start 2024-09-10T20:59:47.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad302bd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad14003a50 0x7fad28153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:47.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad302bd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad28160cc0 0x7fad2814a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:47.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad302bd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad28158000 0x7fad28159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:47.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad302bd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fad280d24e0 con 0x7fad14003a50 2024-09-10T20:59:47.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad302bd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fad280b70b0 con 0x7fad28160cc0 2024-09-10T20:59:47.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad302bd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fad280d2660 con 0x7fad28158000 2024-09-10T20:59:47.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad2e059640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad14003a50 0x7fad28153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:47.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad2e059640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad14003a50 0x7fad28153960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45886/0 (socket says 172.21.15.98:45886) 2024-09-10T20:59:47.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad2e059640 1 -- 172.21.15.98:0/3697115890 learned_addr learned my addr 172.21.15.98:0/3697115890 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:47.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad2e059640 1 -- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad28158000 msgr2=0x7fad28159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad2e85a640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad28158000 0x7fad28159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:47.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad2e059640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad28158000 0x7fad28159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad2d858640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad28160cc0 0x7fad2814a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:47.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad2e059640 1 -- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad28160cc0 msgr2=0x7fad2814a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad2e059640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad28160cc0 0x7fad2814a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad2e059640 1 -- 172.21.15.98:0/3697115890 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad28158580 con 0x7fad14003a50 2024-09-10T20:59:47.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad2d858640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad28160cc0 0x7fad2814a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:47.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.447+0000 7fad2e059640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad14003a50 0x7fad28153960 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fad24034040 tx=0x7fad24003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:47.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.448+0000 7fad1effd640 1 -- 172.21.15.98:0/3697115890 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24005020 con 0x7fad14003a50 2024-09-10T20:59:47.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.448+0000 7fad1effd640 1 -- 172.21.15.98:0/3697115890 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad240052b0 con 0x7fad14003a50 2024-09-10T20:59:47.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.448+0000 7fad302bd640 1 -- 172.21.15.98:0/3697115890 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad281601e0 con 0x7fad14003a50 2024-09-10T20:59:47.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.448+0000 7fad1effd640 1 -- 172.21.15.98:0/3697115890 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad240047a0 con 0x7fad14003a50 2024-09-10T20:59:47.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.448+0000 7fad302bd640 1 -- 172.21.15.98:0/3697115890 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad281607a0 con 0x7fad14003a50 2024-09-10T20:59:47.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.449+0000 7fad302bd640 1 -- 172.21.15.98:0/3697115890 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad280d2b50 con 0x7fad14003a50 2024-09-10T20:59:47.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.449+0000 7fad1effd640 1 -- 172.21.15.98:0/3697115890 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fad24004940 con 0x7fad14003a50 2024-09-10T20:59:47.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.450+0000 7fad1effd640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fad0c077fa0 0x7fad0c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:47.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.450+0000 7fad1effd640 1 -- 172.21.15.98:0/3697115890 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fad240bad90 con 0x7fad14003a50 2024-09-10T20:59:47.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.450+0000 7fad2d858640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fad0c077fa0 0x7fad0c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:47.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.450+0000 7fad2d858640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fad0c077fa0 0x7fad0c07a460 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad18006990 tx=0x7fad18006b00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:47.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.454+0000 7fad1effd640 1 -- 172.21.15.98:0/3697115890 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fad24083140 con 0x7fad14003a50 2024-09-10T20:59:47.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.629+0000 7fad302bd640 1 -- 172.21.15.98:0/3697115890 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad2808dfa0 con 0x7fad14003a50 2024-09-10T20:59:47.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.630+0000 7fad1effd640 1 -- 172.21.15.98:0/3697115890 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17027 (secure 0 0 0) 0x7fad24087b70 con 0x7fad14003a50 2024-09-10T20:59:47.635 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:47.635 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":39,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:59:46.344165+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:59:44.162419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T20:59:45.314201+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:59:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.632+0000 7fad302bd640 1 -- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fad0c077fa0 msgr2=0x7fad0c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.632+0000 7fad302bd640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fad0c077fa0 0x7fad0c07a460 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad18006990 tx=0x7fad18006b00 comp rx=0 tx=0).stop 2024-09-10T20:59:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.632+0000 7fad302bd640 1 -- 172.21.15.98:0/3697115890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad14003a50 msgr2=0x7fad28153960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.632+0000 7fad302bd640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad14003a50 0x7fad28153960 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fad24034040 tx=0x7fad24003880 comp rx=0 tx=0).stop 2024-09-10T20:59:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.632+0000 7fad302bd640 1 -- 172.21.15.98:0/3697115890 shutdown_connections 2024-09-10T20:59:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.632+0000 7fad302bd640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fad0c077fa0 0x7fad0c07a460 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.632+0000 7fad302bd640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad28158000 0x7fad28159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.632+0000 7fad302bd640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad28160cc0 0x7fad2814a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.632+0000 7fad302bd640 1 --2- 172.21.15.98:0/3697115890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad14003a50 0x7fad28153960 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.632+0000 7fad302bd640 1 -- 172.21.15.98:0/3697115890 >> 172.21.15.98:0/3697115890 conn(0x7fad280d3f70 msgr2=0x7fad280cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.632+0000 7fad302bd640 1 -- 172.21.15.98:0/3697115890 shutdown_connections 2024-09-10T20:59:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.632+0000 7fad302bd640 1 -- 172.21.15.98:0/3697115890 wait complete. 2024-09-10T20:59:47.648 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T20:59:47.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.747+0000 7ffa3d565640 1 -- 172.21.15.98:0/558818891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa380d2300 msgr2=0x7ffa380d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.747+0000 7ffa3d565640 1 --2- 172.21.15.98:0/558818891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa380d2300 0x7ffa380d2730 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009830 tx=0x7ffa2c02a480 comp rx=0 tx=0).stop 2024-09-10T20:59:47.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.747+0000 7ffa3d565640 1 -- 172.21.15.98:0/558818891 shutdown_connections 2024-09-10T20:59:47.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.747+0000 7ffa3d565640 1 --2- 172.21.15.98:0/558818891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa380d2300 0x7ffa380d2730 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.747+0000 7ffa3d565640 1 -- 172.21.15.98:0/558818891 >> 172.21.15.98:0/558818891 conn(0x7ffa380d3db0 msgr2=0x7ffa380d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:47.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.747+0000 7ffa3d565640 1 -- 172.21.15.98:0/558818891 shutdown_connections 2024-09-10T20:59:47.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.747+0000 7ffa3d565640 1 -- 172.21.15.98:0/558818891 wait complete. 2024-09-10T20:59:47.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.749+0000 7ffa3d565640 1 Processor -- start 2024-09-10T20:59:47.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.749+0000 7ffa3d565640 1 -- start start 2024-09-10T20:59:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.749+0000 7ffa3d565640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa380b2410 0x7ffa380a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.749+0000 7ffa3d565640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa380d2300 0x7ffa3815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.749+0000 7ffa3d565640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa3815dea0 0x7ffa380cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.749+0000 7ffa3d565640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffa380b6c00 con 0x7ffa380d2300 2024-09-10T20:59:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.749+0000 7ffa3d565640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffa380b6d80 con 0x7ffa3815dea0 2024-09-10T20:59:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.749+0000 7ffa3d565640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffa380b2fb0 con 0x7ffa380b2410 2024-09-10T20:59:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.749+0000 7ffa367fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa380d2300 0x7ffa3815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.749+0000 7ffa367fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa380d2300 0x7ffa3815b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45906/0 (socket says 172.21.15.98:45906) 2024-09-10T20:59:47.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.749+0000 7ffa367fc640 1 -- 172.21.15.98:0/3064718662 learned_addr learned my addr 172.21.15.98:0/3064718662 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:47.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.750+0000 7ffa377fe640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa3815dea0 0x7ffa380cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:47.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.750+0000 7ffa36ffd640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa380b2410 0x7ffa380a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:47.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.750+0000 7ffa367fc640 1 -- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa380b2410 msgr2=0x7ffa380a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.750+0000 7ffa367fc640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa380b2410 0x7ffa380a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.750+0000 7ffa367fc640 1 -- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa3815dea0 msgr2=0x7ffa380cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:47.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.750+0000 7ffa367fc640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa3815dea0 0x7ffa380cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:47.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.750+0000 7ffa367fc640 1 -- 172.21.15.98:0/3064718662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3815b8e0 con 0x7ffa380d2300 2024-09-10T20:59:47.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.750+0000 7ffa36ffd640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa380b2410 0x7ffa380a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:47.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.750+0000 7ffa367fc640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa380d2300 0x7ffa3815b3a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c000c00 tx=0x7ffa2c032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:47.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.750+0000 7ffa1f7fe640 1 -- 172.21.15.98:0/3064718662 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c040070 con 0x7ffa380d2300 2024-09-10T20:59:47.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.750+0000 7ffa1f7fe640 1 -- 172.21.15.98:0/3064718662 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa2c003af0 con 0x7ffa380d2300 2024-09-10T20:59:47.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.750+0000 7ffa1f7fe640 1 -- 172.21.15.98:0/3064718662 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c045460 con 0x7ffa380d2300 2024-09-10T20:59:47.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.750+0000 7ffa3d565640 1 -- 172.21.15.98:0/3064718662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa381f52c0 con 0x7ffa380d2300 2024-09-10T20:59:47.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.750+0000 7ffa3d565640 1 -- 172.21.15.98:0/3064718662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa380cbee0 con 0x7ffa380d2300 2024-09-10T20:59:47.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.751+0000 7ffa3d565640 1 -- 172.21.15.98:0/3064718662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa380d2730 con 0x7ffa380d2300 2024-09-10T20:59:47.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.752+0000 7ffa1f7fe640 1 -- 172.21.15.98:0/3064718662 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ffa2c04d030 con 0x7ffa380d2300 2024-09-10T20:59:47.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.752+0000 7ffa1f7fe640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa24077fa0 0x7ffa2407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:47.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.752+0000 7ffa1f7fe640 1 -- 172.21.15.98:0/3064718662 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ffa2c0bb510 con 0x7ffa380d2300 2024-09-10T20:59:47.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.753+0000 7ffa36ffd640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa24077fa0 0x7ffa2407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:47.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.753+0000 7ffa36ffd640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa24077fa0 0x7ffa2407a460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa3815d510 tx=0x7ffa280073c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:47.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.756+0000 7ffa1f7fe640 1 -- 172.21.15.98:0/3064718662 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ffa2c083e80 con 0x7ffa380d2300 2024-09-10T20:59:47.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:47.967+0000 7ffa3d565640 1 -- 172.21.15.98:0/3064718662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ffa380d02e0 con 0x7ffa380d2300 2024-09-10T20:59:48.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.344+0000 7ffa1f7fe640 1 -- 172.21.15.98:0/3064718662 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7ffa2c083780 con 0x7ffa380d2300 2024-09-10T20:59:48.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.346+0000 7ffa3d565640 1 -- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa24077fa0 msgr2=0x7ffa2407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:48.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.346+0000 7ffa3d565640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa24077fa0 0x7ffa2407a460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa3815d510 tx=0x7ffa280073c0 comp rx=0 tx=0).stop 2024-09-10T20:59:48.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.346+0000 7ffa3d565640 1 -- 172.21.15.98:0/3064718662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa380d2300 msgr2=0x7ffa3815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:48.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.346+0000 7ffa3d565640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa380d2300 0x7ffa3815b3a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c000c00 tx=0x7ffa2c032d50 comp rx=0 tx=0).stop 2024-09-10T20:59:48.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.346+0000 7ffa3d565640 1 -- 172.21.15.98:0/3064718662 shutdown_connections 2024-09-10T20:59:48.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.346+0000 7ffa3d565640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ffa24077fa0 0x7ffa2407a460 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:48.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.346+0000 7ffa3d565640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffa3815dea0 0x7ffa380cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:48.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.346+0000 7ffa3d565640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa380d2300 0x7ffa3815b3a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:48.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.346+0000 7ffa3d565640 1 --2- 172.21.15.98:0/3064718662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffa380b2410 0x7ffa380a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:48.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.346+0000 7ffa3d565640 1 -- 172.21.15.98:0/3064718662 >> 172.21.15.98:0/3064718662 conn(0x7ffa380d3db0 msgr2=0x7ffa380d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:48.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.346+0000 7ffa3d565640 1 -- 172.21.15.98:0/3064718662 shutdown_connections 2024-09-10T20:59:48.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.346+0000 7ffa3d565640 1 -- 172.21.15.98:0/3064718662 wait complete. 2024-09-10T20:59:48.361 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T20:59:48.362 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T20:59:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.462+0000 7ff93787f640 1 -- 172.21.15.98:0/133309051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9300d4af0 msgr2=0x7ff9300d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.462+0000 7ff93787f640 1 --2- 172.21.15.98:0/133309051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9300d4af0 0x7ff9300d24e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff920009860 tx=0x7ff92002a4b0 comp rx=0 tx=0).stop 2024-09-10T20:59:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.462+0000 7ff93787f640 1 -- 172.21.15.98:0/133309051 shutdown_connections 2024-09-10T20:59:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.462+0000 7ff93787f640 1 --2- 172.21.15.98:0/133309051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9300d4af0 0x7ff9300d24e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.462+0000 7ff93787f640 1 -- 172.21.15.98:0/133309051 >> 172.21.15.98:0/133309051 conn(0x7ff9300d3f70 msgr2=0x7ff9300d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.462+0000 7ff93787f640 1 -- 172.21.15.98:0/133309051 shutdown_connections 2024-09-10T20:59:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.462+0000 7ff93787f640 1 -- 172.21.15.98:0/133309051 wait complete. 2024-09-10T20:59:48.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.463+0000 7ff93787f640 1 Processor -- start 2024-09-10T20:59:48.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.464+0000 7ff93787f640 1 -- start start 2024-09-10T20:59:48.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.464+0000 7ff93787f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9300d2a20 0x7ff9301556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.464+0000 7ff93787f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9300d4af0 0x7ff930160860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.464+0000 7ff93787f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff93015e060 0x7ff9300c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.464+0000 7ff93787f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff9300cf2d0 con 0x7ff9300d4af0 2024-09-10T20:59:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.464+0000 7ff93787f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff9300cf150 con 0x7ff93015e060 2024-09-10T20:59:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.464+0000 7ff93787f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff9300cf450 con 0x7ff9300d2a20 2024-09-10T20:59:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.464+0000 7ff934e1a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9300d4af0 0x7ff930160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.464+0000 7ff934e1a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9300d4af0 0x7ff930160860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45920/0 (socket says 172.21.15.98:45920) 2024-09-10T20:59:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.464+0000 7ff934e1a640 1 -- 172.21.15.98:0/4029357706 learned_addr learned my addr 172.21.15.98:0/4029357706 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.465+0000 7ff93561b640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9300d2a20 0x7ff9301556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.465+0000 7ff934e1a640 1 -- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9300d2a20 msgr2=0x7ff9301556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.465+0000 7ff935e1c640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff93015e060 0x7ff9300c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.465+0000 7ff934e1a640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9300d2a20 0x7ff9301556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.465+0000 7ff934e1a640 1 -- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff93015e060 msgr2=0x7ff9300c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.465+0000 7ff934e1a640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff93015e060 0x7ff9300c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.465+0000 7ff934e1a640 1 -- 172.21.15.98:0/4029357706 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9300c89a0 con 0x7ff9300d4af0 2024-09-10T20:59:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.465+0000 7ff935e1c640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff93015e060 0x7ff9300c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.465+0000 7ff934e1a640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9300d4af0 0x7ff930160860 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff92002aaa0 tx=0x7ff920032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.465+0000 7ff925ffb640 1 -- 172.21.15.98:0/4029357706 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920040070 con 0x7ff9300d4af0 2024-09-10T20:59:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.465+0000 7ff925ffb640 1 -- 172.21.15.98:0/4029357706 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff920003af0 con 0x7ff9300d4af0 2024-09-10T20:59:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.465+0000 7ff93787f640 1 -- 172.21.15.98:0/4029357706 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9300c8c80 con 0x7ff9300d4af0 2024-09-10T20:59:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.465+0000 7ff925ffb640 1 -- 172.21.15.98:0/4029357706 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920045460 con 0x7ff9300d4af0 2024-09-10T20:59:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.465+0000 7ff93787f640 1 -- 172.21.15.98:0/4029357706 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9300c9240 con 0x7ff9300d4af0 2024-09-10T20:59:48.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.466+0000 7ff93787f640 1 -- 172.21.15.98:0/4029357706 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9300c9580 con 0x7ff9300d4af0 2024-09-10T20:59:48.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.467+0000 7ff925ffb640 1 -- 172.21.15.98:0/4029357706 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff92004d030 con 0x7ff9300d4af0 2024-09-10T20:59:48.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.467+0000 7ff925ffb640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff914077f70 0x7ff91407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:48.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.467+0000 7ff925ffb640 1 -- 172.21.15.98:0/4029357706 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ff9200bb480 con 0x7ff9300d4af0 2024-09-10T20:59:48.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.467+0000 7ff93561b640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff914077f70 0x7ff91407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:48.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.468+0000 7ff93561b640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff914077f70 0x7ff91407a430 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00a730 tx=0x7ff92c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:48.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.471+0000 7ff925ffb640 1 -- 172.21.15.98:0/4029357706 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff9300c9580 con 0x7ff9300d4af0 2024-09-10T20:59:48.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:48.681+0000 7ff93787f640 1 -- 172.21.15.98:0/4029357706 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff9300d0100 con 0x7ff9300d4af0 2024-09-10T20:59:49.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.348+0000 7ff925ffb640 1 -- 172.21.15.98:0/4029357706 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7ff920083e80 con 0x7ff9300d4af0 2024-09-10T20:59:49.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.352+0000 7ff93787f640 1 -- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff914077f70 msgr2=0x7ff91407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:49.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.352+0000 7ff93787f640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff914077f70 0x7ff91407a430 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00a730 tx=0x7ff92c008040 comp rx=0 tx=0).stop 2024-09-10T20:59:49.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.352+0000 7ff93787f640 1 -- 172.21.15.98:0/4029357706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9300d4af0 msgr2=0x7ff930160860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:49.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.352+0000 7ff93787f640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9300d4af0 0x7ff930160860 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff92002aaa0 tx=0x7ff920032d50 comp rx=0 tx=0).stop 2024-09-10T20:59:49.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.352+0000 7ff93787f640 1 -- 172.21.15.98:0/4029357706 shutdown_connections 2024-09-10T20:59:49.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.352+0000 7ff93787f640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff914077f70 0x7ff91407a430 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.352+0000 7ff93787f640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff93015e060 0x7ff9300c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.352+0000 7ff93787f640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9300d4af0 0x7ff930160860 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.352+0000 7ff93787f640 1 --2- 172.21.15.98:0/4029357706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9300d2a20 0x7ff9301556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.352+0000 7ff93787f640 1 -- 172.21.15.98:0/4029357706 >> 172.21.15.98:0/4029357706 conn(0x7ff9300d3f70 msgr2=0x7ff93006e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:49.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.352+0000 7ff93787f640 1 -- 172.21.15.98:0/4029357706 shutdown_connections 2024-09-10T20:59:49.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.352+0000 7ff93787f640 1 -- 172.21.15.98:0/4029357706 wait complete. 2024-09-10T20:59:49.369 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T20:59:49.369 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:59:49.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.471+0000 7f098d9a1640 1 -- 172.21.15.98:0/3829686650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0988154ee0 msgr2=0x7f098815b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:49.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.471+0000 7f098d9a1640 1 --2- 172.21.15.98:0/3829686650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0988154ee0 0x7f098815b850 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f097c009830 tx=0x7f097c02a510 comp rx=0 tx=0).stop 2024-09-10T20:59:49.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.471+0000 7f098d9a1640 1 -- 172.21.15.98:0/3829686650 shutdown_connections 2024-09-10T20:59:49.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.471+0000 7f098d9a1640 1 --2- 172.21.15.98:0/3829686650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0988070dd0 0x7f09880cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.471+0000 7f098d9a1640 1 --2- 172.21.15.98:0/3829686650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0988154ee0 0x7f098815b850 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.471+0000 7f098d9a1640 1 -- 172.21.15.98:0/3829686650 >> 172.21.15.98:0/3829686650 conn(0x7f09880d09f0 msgr2=0x7f09880d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:49.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.471+0000 7f098d9a1640 1 -- 172.21.15.98:0/3829686650 shutdown_connections 2024-09-10T20:59:49.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.471+0000 7f098d9a1640 1 -- 172.21.15.98:0/3829686650 wait complete. 2024-09-10T20:59:49.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.473+0000 7f098d9a1640 1 Processor -- start 2024-09-10T20:59:49.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.473+0000 7f098d9a1640 1 -- start start 2024-09-10T20:59:49.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.473+0000 7f098d9a1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0988070dd0 0x7f0988150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:49.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.473+0000 7f098d9a1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0988154ee0 0x7f09880cf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:49.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.473+0000 7f098d9a1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f098815be70 0x7f0988159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:49.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.473+0000 7f098d9a1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f09880b5080 con 0x7f0988070dd0 2024-09-10T20:59:49.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.473+0000 7f098d9a1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f09880b4f00 con 0x7f0988154ee0 2024-09-10T20:59:49.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.473+0000 7f098d9a1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f09880b5200 con 0x7f098815be70 2024-09-10T20:59:49.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f098c99f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0988070dd0 0x7f0988150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:49.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f098c99f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0988070dd0 0x7f0988150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45922/0 (socket says 172.21.15.98:45922) 2024-09-10T20:59:49.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f098c99f640 1 -- 172.21.15.98:0/151141810 learned_addr learned my addr 172.21.15.98:0/151141810 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:49.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f098d1a0640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f098815be70 0x7f0988159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:49.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f0987fff640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0988154ee0 0x7f09880cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:49.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f098c99f640 1 -- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f098815be70 msgr2=0x7f0988159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:49.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f098c99f640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f098815be70 0x7f0988159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f098c99f640 1 -- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0988154ee0 msgr2=0x7f09880cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:49.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f098c99f640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0988154ee0 0x7f09880cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f098c99f640 1 -- 172.21.15.98:0/151141810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09881f5400 con 0x7f0988070dd0 2024-09-10T20:59:49.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f0987fff640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0988154ee0 0x7f09880cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:49.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f098c99f640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0988070dd0 0x7f0988150d70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f097c02ab00 tx=0x7f097c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:49.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f09857fa640 1 -- 172.21.15.98:0/151141810 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c040070 con 0x7f0988070dd0 2024-09-10T20:59:49.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f09857fa640 1 -- 172.21.15.98:0/151141810 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f097c005310 con 0x7f0988070dd0 2024-09-10T20:59:49.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f098d9a1640 1 -- 172.21.15.98:0/151141810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09881f4640 con 0x7f0988070dd0 2024-09-10T20:59:49.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f09857fa640 1 -- 172.21.15.98:0/151141810 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c0047a0 con 0x7f0988070dd0 2024-09-10T20:59:49.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.474+0000 7f098d9a1640 1 -- 172.21.15.98:0/151141810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09881f4c00 con 0x7f0988070dd0 2024-09-10T20:59:49.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.475+0000 7f098d9a1640 1 -- 172.21.15.98:0/151141810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09880b5720 con 0x7f0988070dd0 2024-09-10T20:59:49.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.476+0000 7f09857fa640 1 -- 172.21.15.98:0/151141810 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f097c004b50 con 0x7f0988070dd0 2024-09-10T20:59:49.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.476+0000 7f09857fa640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f096c077f70 0x7f096c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:49.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.476+0000 7f09857fa640 1 -- 172.21.15.98:0/151141810 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f097c0bb450 con 0x7f0988070dd0 2024-09-10T20:59:49.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.477+0000 7f0987fff640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f096c077f70 0x7f096c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:49.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.477+0000 7f0987fff640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f096c077f70 0x7f096c07a430 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0974007fd0 tx=0x7f0974009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:49.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.480+0000 7f09857fa640 1 -- 172.21.15.98:0/151141810 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f097c083dd0 con 0x7f0988070dd0 2024-09-10T20:59:49.610 INFO:tasks.ceph.mds.a.smithi098.stderr:2024-09-10T20:59:49.606+0000 7f3e137c8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T20:59:49.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.688+0000 7f098d9a1640 1 -- 172.21.15.98:0/151141810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09880b5990 con 0x7f0988070dd0 2024-09-10T20:59:49.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.689+0000 7f09857fa640 1 -- 172.21.15.98:0/151141810 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7f097c0836d0 con 0x7f0988070dd0 2024-09-10T20:59:49.693 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:49.693 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":17,"btime":"2024-09-10T20:59:49:345393+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/1700293059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":1700293059},{"type":"v1","addr":"172.21.15.98:6834","nonce":1700293059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4048376378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4048376378},{"type":"v1","addr":"172.21.15.190:6835","nonce":4048376378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3087107175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3087107175},{"type":"v1","addr":"172.21.15.190:6837","nonce":3087107175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:59:46.344123+0000","modified":"2024-09-10T20:59:48.682750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5001},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5001":{"gid":5001,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.98:6835/4202850803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":4202850803},{"type":"v1","addr":"172.21.15.98:6835","nonce":4202850803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5001,"qdb_cluster":[5001]},"id":2}]} 2024-09-10T20:59:49.694 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 17 2024-09-10T20:59:49.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.691+0000 7f098d9a1640 1 -- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f096c077f70 msgr2=0x7f096c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:49.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.691+0000 7f098d9a1640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f096c077f70 0x7f096c07a430 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0974007fd0 tx=0x7f0974009040 comp rx=0 tx=0).stop 2024-09-10T20:59:49.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.691+0000 7f098d9a1640 1 -- 172.21.15.98:0/151141810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0988070dd0 msgr2=0x7f0988150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:49.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.691+0000 7f098d9a1640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0988070dd0 0x7f0988150d70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f097c02ab00 tx=0x7f097c032c10 comp rx=0 tx=0).stop 2024-09-10T20:59:49.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.691+0000 7f098d9a1640 1 -- 172.21.15.98:0/151141810 shutdown_connections 2024-09-10T20:59:49.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.691+0000 7f098d9a1640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f096c077f70 0x7f096c07a430 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.691+0000 7f098d9a1640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f098815be70 0x7f0988159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.691+0000 7f098d9a1640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0988154ee0 0x7f09880cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.691+0000 7f098d9a1640 1 --2- 172.21.15.98:0/151141810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0988070dd0 0x7f0988150d70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.691+0000 7f098d9a1640 1 -- 172.21.15.98:0/151141810 >> 172.21.15.98:0/151141810 conn(0x7f09880d09f0 msgr2=0x7f098806faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:49.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.691+0000 7f098d9a1640 1 -- 172.21.15.98:0/151141810 shutdown_connections 2024-09-10T20:59:49.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.691+0000 7f098d9a1640 1 -- 172.21.15.98:0/151141810 wait complete. 2024-09-10T20:59:49.706 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T20:59:49.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.803+0000 7f84aa07f640 1 -- 172.21.15.98:0/2429236896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a415e060 msgr2=0x7f84a40b6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:49.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.803+0000 7f84aa07f640 1 --2- 172.21.15.98:0/2429236896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a415e060 0x7f84a40b6dc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f84a408c0d0 tx=0x7f849402a580 comp rx=0 tx=0).stop 2024-09-10T20:59:49.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.803+0000 7f84aa07f640 1 -- 172.21.15.98:0/2429236896 shutdown_connections 2024-09-10T20:59:49.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.803+0000 7f84aa07f640 1 --2- 172.21.15.98:0/2429236896 >> v2:172.21.15.190:3300/0 conn(0x7f84a40d4af0 0x7f84a40d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.803+0000 7f84aa07f640 1 --2- 172.21.15.98:0/2429236896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84a40b7300 0x7f84a40d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.803+0000 7f84aa07f640 1 --2- 172.21.15.98:0/2429236896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a415e060 0x7f84a40b6dc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.803+0000 7f84aa07f640 1 -- 172.21.15.98:0/2429236896 >> 172.21.15.98:0/2429236896 conn(0x7f84a40d3f70 msgr2=0x7f84a40d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:49.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.803+0000 7f84aa07f640 1 -- 172.21.15.98:0/2429236896 shutdown_connections 2024-09-10T20:59:49.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.803+0000 7f84aa07f640 1 -- 172.21.15.98:0/2429236896 wait complete. 2024-09-10T20:59:49.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.804+0000 7f84aa07f640 1 Processor -- start 2024-09-10T20:59:49.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.805+0000 7f84aa07f640 1 -- start start 2024-09-10T20:59:49.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.805+0000 7f84aa07f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84a40d4af0 0x7f84a4153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:49.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.805+0000 7f84aa07f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a415e060 0x7f84a41f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:49.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.805+0000 7f84aa07f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84a41f4f70 0x7f84a40cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:49.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.805+0000 7f84aa07f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4161e60 con 0x7f84a415e060 2024-09-10T20:59:49.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.805+0000 7f84aa07f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4161ce0 con 0x7f84a40d4af0 2024-09-10T20:59:49.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.805+0000 7f84aa07f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f84a4161fe0 con 0x7f84a41f4f70 2024-09-10T20:59:49.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.805+0000 7f849bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a415e060 0x7f84a41f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:49.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.805+0000 7f849bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a415e060 0x7f84a41f4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45936/0 (socket says 172.21.15.98:45936) 2024-09-10T20:59:49.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.805+0000 7f849bfff640 1 -- 172.21.15.98:0/479589925 learned_addr learned my addr 172.21.15.98:0/479589925 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:49.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.805+0000 7f84a37fe640 1 --2- 172.21.15.98:0/479589925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84a40d4af0 0x7f84a4153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:49.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.805+0000 7f84a3fff640 1 --2- 172.21.15.98:0/479589925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84a41f4f70 0x7f84a40cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:49.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.806+0000 7f849bfff640 1 -- 172.21.15.98:0/479589925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84a41f4f70 msgr2=0x7f84a40cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:49.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.806+0000 7f849bfff640 1 --2- 172.21.15.98:0/479589925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84a41f4f70 0x7f84a40cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.806+0000 7f849bfff640 1 -- 172.21.15.98:0/479589925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84a40d4af0 msgr2=0x7f84a4153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:49.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.806+0000 7f849bfff640 1 --2- 172.21.15.98:0/479589925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84a40d4af0 0x7f84a4153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.806+0000 7f849bfff640 1 -- 172.21.15.98:0/479589925 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a4153c60 con 0x7f84a415e060 2024-09-10T20:59:49.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.806+0000 7f849bfff640 1 --2- 172.21.15.98:0/479589925 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a415e060 0x7f84a41f4a30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f848c00d770 tx=0x7f848c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:49.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.806+0000 7f84a0ff9640 1 -- 172.21.15.98:0/479589925 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c017070 con 0x7f84a415e060 2024-09-10T20:59:49.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.806+0000 7f84a0ff9640 1 -- 172.21.15.98:0/479589925 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f848c009d20 con 0x7f84a415e060 2024-09-10T20:59:49.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.806+0000 7f84aa07f640 1 -- 172.21.15.98:0/479589925 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a415f1f0 con 0x7f84a415e060 2024-09-10T20:59:49.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.806+0000 7f84a0ff9640 1 -- 172.21.15.98:0/479589925 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c005280 con 0x7f84a415e060 2024-09-10T20:59:49.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.806+0000 7f84aa07f640 1 -- 172.21.15.98:0/479589925 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a41f5540 con 0x7f84a415e060 2024-09-10T20:59:49.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.807+0000 7f84aa07f640 1 -- 172.21.15.98:0/479589925 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a40b6dc0 con 0x7f84a415e060 2024-09-10T20:59:49.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.808+0000 7f84a0ff9640 1 -- 172.21.15.98:0/479589925 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f848c005420 con 0x7f84a415e060 2024-09-10T20:59:49.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.808+0000 7f84a0ff9640 1 --2- 172.21.15.98:0/479589925 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8484077f70 0x7f848407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:49.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.808+0000 7f84a0ff9640 1 -- 172.21.15.98:0/479589925 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f848c09a0a0 con 0x7f84a415e060 2024-09-10T20:59:49.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.808+0000 7f84a37fe640 1 --2- 172.21.15.98:0/479589925 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8484077f70 0x7f848407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:49.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.809+0000 7f84a37fe640 1 --2- 172.21.15.98:0/479589925 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8484077f70 0x7f848407a430 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8494033000 tx=0x7f8494004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:49.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.812+0000 7f84a0ff9640 1 -- 172.21.15.98:0/479589925 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f848c0623c0 con 0x7f84a415e060 2024-09-10T20:59:49.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.987+0000 7f84aa07f640 1 -- 172.21.15.98:0/479589925 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84a40cec50 con 0x7f84a415e060 2024-09-10T20:59:49.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.988+0000 7f84a0ff9640 1 -- 172.21.15.98:0/479589925 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17014 (secure 0 0 0) 0x7f848c06b130 con 0x7f84a415e060 2024-09-10T20:59:49.992 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:49.992 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":40,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T20:59:48.004958+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:59:44.162419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T20:59:45.314201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T20:59:49.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.990+0000 7f84aa07f640 1 -- 172.21.15.98:0/479589925 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8484077f70 msgr2=0x7f848407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:49.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.990+0000 7f84aa07f640 1 --2- 172.21.15.98:0/479589925 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8484077f70 0x7f848407a430 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8494033000 tx=0x7f8494004730 comp rx=0 tx=0).stop 2024-09-10T20:59:49.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.990+0000 7f84aa07f640 1 -- 172.21.15.98:0/479589925 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a415e060 msgr2=0x7f84a41f4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:49.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.990+0000 7f84aa07f640 1 --2- 172.21.15.98:0/479589925 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a415e060 0x7f84a41f4a30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f848c00d770 tx=0x7f848c00dc30 comp rx=0 tx=0).stop 2024-09-10T20:59:49.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.990+0000 7f84aa07f640 1 -- 172.21.15.98:0/479589925 shutdown_connections 2024-09-10T20:59:49.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.990+0000 7f84aa07f640 1 --2- 172.21.15.98:0/479589925 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8484077f70 0x7f848407a430 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.990+0000 7f84aa07f640 1 --2- 172.21.15.98:0/479589925 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84a41f4f70 0x7f84a40cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.990+0000 7f84aa07f640 1 --2- 172.21.15.98:0/479589925 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84a415e060 0x7f84a41f4a30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.990+0000 7f84aa07f640 1 --2- 172.21.15.98:0/479589925 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84a40d4af0 0x7f84a4153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:49.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.990+0000 7f84aa07f640 1 -- 172.21.15.98:0/479589925 >> 172.21.15.98:0/479589925 conn(0x7f84a40d3f70 msgr2=0x7f84a40b7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:49.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.990+0000 7f84aa07f640 1 -- 172.21.15.98:0/479589925 shutdown_connections 2024-09-10T20:59:49.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:49.990+0000 7f84aa07f640 1 -- 172.21.15.98:0/479589925 wait complete. 2024-09-10T20:59:50.006 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T20:59:50.006 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:59:50.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.104+0000 7f6f7027f640 1 -- 172.21.15.98:0/283791439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f54003a50 msgr2=0x7f6f54003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.104+0000 7f6f7027f640 1 --2- 172.21.15.98:0/283791439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f54003a50 0x7f6f54003e70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009b20 tx=0x7f6f54005830 comp rx=0 tx=0).stop 2024-09-10T20:59:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.104+0000 7f6f7027f640 1 -- 172.21.15.98:0/283791439 shutdown_connections 2024-09-10T20:59:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.104+0000 7f6f7027f640 1 --2- 172.21.15.98:0/283791439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f54003a50 0x7f6f54003e70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.104+0000 7f6f7027f640 1 -- 172.21.15.98:0/283791439 >> 172.21.15.98:0/283791439 conn(0x7f6f680b1760 msgr2=0x7f6f680b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.104+0000 7f6f7027f640 1 -- 172.21.15.98:0/283791439 shutdown_connections 2024-09-10T20:59:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.104+0000 7f6f7027f640 1 -- 172.21.15.98:0/283791439 wait complete. 2024-09-10T20:59:50.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.105+0000 7f6f7027f640 1 Processor -- start 2024-09-10T20:59:50.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.105+0000 7f6f7027f640 1 -- start start 2024-09-10T20:59:50.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f7027f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f54003a50 0x7f6f6813cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:50.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f7027f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f681e7be0 0x7f6f681e6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:50.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f7027f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f681e65a0 0x7f6f680b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:50.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f7027f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6f680cec80 con 0x7f6f681e65a0 2024-09-10T20:59:50.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f7027f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6f680ceb00 con 0x7f6f681e7be0 2024-09-10T20:59:50.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f7027f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6f680cee00 con 0x7f6f54003a50 2024-09-10T20:59:50.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f6e81c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f681e65a0 0x7f6f680b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:50.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f6e81c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f681e65a0 0x7f6f680b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45960/0 (socket says 172.21.15.98:45960) 2024-09-10T20:59:50.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f6e81c640 1 -- 172.21.15.98:0/348641839 learned_addr learned my addr 172.21.15.98:0/348641839 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:50.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f6d81a640 1 --2- 172.21.15.98:0/348641839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f681e7be0 0x7f6f681e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:50.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f6e01b640 1 --2- 172.21.15.98:0/348641839 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f54003a50 0x7f6f6813cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:50.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f6e81c640 1 -- 172.21.15.98:0/348641839 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f54003a50 msgr2=0x7f6f6813cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f6e81c640 1 --2- 172.21.15.98:0/348641839 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f54003a50 0x7f6f6813cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f6e81c640 1 -- 172.21.15.98:0/348641839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f681e7be0 msgr2=0x7f6f681e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f6e81c640 1 --2- 172.21.15.98:0/348641839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f681e7be0 0x7f6f681e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f6e81c640 1 -- 172.21.15.98:0/348641839 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f681e6b40 con 0x7f6f681e65a0 2024-09-10T20:59:50.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.106+0000 7f6f6e81c640 1 --2- 172.21.15.98:0/348641839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f681e65a0 0x7f6f680b0800 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f6815bfd0 tx=0x7f6f6000dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.107+0000 7f6f5affd640 1 -- 172.21.15.98:0/348641839 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60003d80 con 0x7f6f681e65a0 2024-09-10T20:59:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.107+0000 7f6f5affd640 1 -- 172.21.15.98:0/348641839 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f60006070 con 0x7f6f681e65a0 2024-09-10T20:59:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.107+0000 7f6f7027f640 1 -- 172.21.15.98:0/348641839 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f681e6e20 con 0x7f6f681e65a0 2024-09-10T20:59:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.107+0000 7f6f5affd640 1 -- 172.21.15.98:0/348641839 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60006480 con 0x7f6f681e65a0 2024-09-10T20:59:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.107+0000 7f6f7027f640 1 -- 172.21.15.98:0/348641839 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f681e73e0 con 0x7f6f681e65a0 2024-09-10T20:59:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.108+0000 7f6f7027f640 1 -- 172.21.15.98:0/348641839 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6815bdf0 con 0x7f6f681e65a0 2024-09-10T20:59:50.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.108+0000 7f6f5affd640 1 -- 172.21.15.98:0/348641839 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6f60006620 con 0x7f6f681e65a0 2024-09-10T20:59:50.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.109+0000 7f6f5affd640 1 --2- 172.21.15.98:0/348641839 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6f4c077f70 0x7f6f4c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:50.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.109+0000 7f6f5affd640 1 -- 172.21.15.98:0/348641839 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f6f60099f70 con 0x7f6f681e65a0 2024-09-10T20:59:50.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.109+0000 7f6f6e01b640 1 --2- 172.21.15.98:0/348641839 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6f4c077f70 0x7f6f4c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:50.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.110+0000 7f6f6e01b640 1 --2- 172.21.15.98:0/348641839 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6f4c077f70 0x7f6f4c07a430 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6f64035000 tx=0x7f6f640046c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:50.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.113+0000 7f6f5affd640 1 -- 172.21.15.98:0/348641839 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6f6815bdf0 con 0x7f6f681e65a0 2024-09-10T20:59:50.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.282+0000 7f6f7027f640 1 -- 172.21.15.98:0/348641839 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f68141270 con 0x7f6f4c077f70 2024-09-10T20:59:50.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.287+0000 7f6f5affd640 1 -- 172.21.15.98:0/348641839 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+238167 (secure 0 0 0) 0x7f6f68141270 con 0x7f6f4c077f70 2024-09-10T20:59:50.295 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:50.296 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-10T20:59:50.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.290+0000 7f6f7027f640 1 -- 172.21.15.98:0/348641839 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6f4c077f70 msgr2=0x7f6f4c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.290+0000 7f6f7027f640 1 --2- 172.21.15.98:0/348641839 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6f4c077f70 0x7f6f4c07a430 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6f64035000 tx=0x7f6f640046c0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.290+0000 7f6f7027f640 1 -- 172.21.15.98:0/348641839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f681e65a0 msgr2=0x7f6f680b0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.290+0000 7f6f7027f640 1 --2- 172.21.15.98:0/348641839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f681e65a0 0x7f6f680b0800 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f6815bfd0 tx=0x7f6f6000dad0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.290+0000 7f6f7027f640 1 -- 172.21.15.98:0/348641839 shutdown_connections 2024-09-10T20:59:50.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.290+0000 7f6f7027f640 1 --2- 172.21.15.98:0/348641839 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6f4c077f70 0x7f6f4c07a430 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.290+0000 7f6f7027f640 1 --2- 172.21.15.98:0/348641839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f681e65a0 0x7f6f680b0800 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.290+0000 7f6f7027f640 1 --2- 172.21.15.98:0/348641839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f681e7be0 0x7f6f681e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.290+0000 7f6f7027f640 1 --2- 172.21.15.98:0/348641839 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f54003a50 0x7f6f6813cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.291+0000 7f6f7027f640 1 -- 172.21.15.98:0/348641839 >> 172.21.15.98:0/348641839 conn(0x7f6f680b1760 msgr2=0x7f6f680b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:50.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.291+0000 7f6f7027f640 1 -- 172.21.15.98:0/348641839 shutdown_connections 2024-09-10T20:59:50.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.291+0000 7f6f7027f640 1 -- 172.21.15.98:0/348641839 wait complete. 2024-09-10T20:59:50.305 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-10T20:59:48.921015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":24,"acting":24,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":220604,"kb_used_data":4364,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754116,"statfs":{"total":773094113280,"available":772868214784,"internally_reserved":0,"allocated":4468736,"data_stored":2569899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000911"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.997501+0000","last_change":"2024-09-10T20:59:44.997501+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.997501+0000","last_undegraded":"2024-09-10T20:59:44.997501+0000","last_fullsized":"2024-09-10T20:59:44.997501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.995186+0000","last_change":"2024-09-10T20:59:44.995186+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.995186+0000","last_undegraded":"2024-09-10T20:59:44.995186+0000","last_fullsized":"2024-09-10T20:59:44.995186+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.999305+0000","last_change":"2024-09-10T20:59:44.999305+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.999305+0000","last_undegraded":"2024-09-10T20:59:44.999305+0000","last_fullsized":"2024-09-10T20:59:44.999305+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.994818+0000","last_change":"2024-09-10T20:59:44.994818+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.994818+0000","last_undegraded":"2024-09-10T20:59:44.994818+0000","last_fullsized":"2024-09-10T20:59:44.994818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.997501+0000","last_change":"2024-09-10T20:59:44.997501+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.997501+0000","last_undegraded":"2024-09-10T20:59:44.997501+0000","last_fullsized":"2024-09-10T20:59:44.997501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.998837+0000","last_change":"2024-09-10T20:59:44.998837+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.998837+0000","last_undegraded":"2024-09-10T20:59:44.998837+0000","last_fullsized":"2024-09-10T20:59:44.998837+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.995492+0000","last_change":"2024-09-10T20:59:44.995492+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.995492+0000","last_undegraded":"2024-09-10T20:59:44.995492+0000","last_fullsized":"2024-09-10T20:59:44.995492+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.998868+0000","last_change":"2024-09-10T20:59:44.998868+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.998868+0000","last_undegraded":"2024-09-10T20:59:44.998868+0000","last_fullsized":"2024-09-10T20:59:44.998868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.997504+0000","last_change":"2024-09-10T20:59:44.997504+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.997504+0000","last_undegraded":"2024-09-10T20:59:44.997504+0000","last_fullsized":"2024-09-10T20:59:44.997504+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.995527+0000","last_change":"2024-09-10T20:59:44.995527+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.995527+0000","last_undegraded":"2024-09-10T20:59:44.995527+0000","last_fullsized":"2024-09-10T20:59:44.995527+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.998797+0000","last_change":"2024-09-10T20:59:44.998797+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.998797+0000","last_undegraded":"2024-09-10T20:59:44.998797+0000","last_fullsized":"2024-09-10T20:59:44.998797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":140,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T20:59:44.999128+0000","last_change":"2024-09-10T20:58:36.895993+0000","last_active":"2024-09-10T20:59:44.999128+0000","last_peered":"2024-09-10T20:59:44.999128+0000","last_clean":"2024-09-10T20:59:44.999128+0000","last_became_active":"2024-09-10T20:58:36.894988+0000","last_became_peered":"2024-09-10T20:58:36.894988+0000","last_unstale":"2024-09-10T20:59:44.999128+0000","last_undegraded":"2024-09-10T20:59:44.999128+0000","last_fullsized":"2024-09-10T20:59:44.999128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_clean_scrub_stamp":"2024-09-10T20:58:35.588702+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.082528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":22,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":767894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":177526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":177526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":171317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":171317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":171317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":999424,"data_stored":761685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":171317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:59:50.310 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:59:50.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.443+0000 7f886c116640 1 -- 172.21.15.98:0/3679750910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f886406f140 msgr2=0x7f886406f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.443+0000 7f886c116640 1 --2- 172.21.15.98:0/3679750910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f886406f140 0x7f886406f5f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f885c005fd0 tx=0x7f885c02aa80 comp rx=0 tx=0).stop 2024-09-10T20:59:50.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.443+0000 7f886c116640 1 -- 172.21.15.98:0/3679750910 shutdown_connections 2024-09-10T20:59:50.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.443+0000 7f886c116640 1 --2- 172.21.15.98:0/3679750910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f886406f140 0x7f886406f5f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.443+0000 7f886c116640 1 --2- 172.21.15.98:0/3679750910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8864073780 0x7f8864073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.443+0000 7f886c116640 1 -- 172.21.15.98:0/3679750910 >> 172.21.15.98:0/3679750910 conn(0x7f88640b1760 msgr2=0x7f88640b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:50.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.443+0000 7f886c116640 1 -- 172.21.15.98:0/3679750910 shutdown_connections 2024-09-10T20:59:50.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.443+0000 7f886c116640 1 -- 172.21.15.98:0/3679750910 wait complete. 2024-09-10T20:59:50.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.444+0000 7f886c116640 1 Processor -- start 2024-09-10T20:59:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.444+0000 7f886c116640 1 -- start start 2024-09-10T20:59:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f886c116640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f886406f140 0x7f88641f5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f886c116640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8864073780 0x7f8864143340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f886c116640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f886415c470 0x7f8864143880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f886c116640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f886406fcb0 con 0x7f886406f140 2024-09-10T20:59:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f886c116640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f886406fb30 con 0x7f8864073780 2024-09-10T20:59:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f886c116640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f886406fe30 con 0x7f886415c470 2024-09-10T20:59:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f8869eb2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f886406f140 0x7f88641f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f8869eb2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f886406f140 0x7f88641f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54814/0 (socket says 172.21.15.98:54814) 2024-09-10T20:59:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f8869eb2640 1 -- 172.21.15.98:0/3908027984 learned_addr learned my addr 172.21.15.98:0/3908027984 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f88696b1640 1 --2- 172.21.15.98:0/3908027984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8864073780 0x7f8864143340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f886a6b3640 1 --2- 172.21.15.98:0/3908027984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f886415c470 0x7f8864143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f8869eb2640 1 -- 172.21.15.98:0/3908027984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f886415c470 msgr2=0x7f8864143880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f8869eb2640 1 --2- 172.21.15.98:0/3908027984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f886415c470 0x7f8864143880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f8869eb2640 1 -- 172.21.15.98:0/3908027984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8864073780 msgr2=0x7f8864143340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f8869eb2640 1 --2- 172.21.15.98:0/3908027984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8864073780 0x7f8864143340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f8869eb2640 1 -- 172.21.15.98:0/3908027984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8864155870 con 0x7f886406f140 2024-09-10T20:59:50.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f8869eb2640 1 --2- 172.21.15.98:0/3908027984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f886406f140 0x7f88641f5460 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f886000d950 tx=0x7f886000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:50.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f885a7fc640 1 -- 172.21.15.98:0/3908027984 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860017070 con 0x7f886406f140 2024-09-10T20:59:50.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f885a7fc640 1 -- 172.21.15.98:0/3908027984 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8860009d20 con 0x7f886406f140 2024-09-10T20:59:50.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f886c116640 1 -- 172.21.15.98:0/3908027984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8864155b50 con 0x7f886406f140 2024-09-10T20:59:50.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.445+0000 7f885a7fc640 1 -- 172.21.15.98:0/3908027984 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88600106c0 con 0x7f886406f140 2024-09-10T20:59:50.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.446+0000 7f886c116640 1 -- 172.21.15.98:0/3908027984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88641f59a0 con 0x7f886406f140 2024-09-10T20:59:50.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.446+0000 7f886c116640 1 -- 172.21.15.98:0/3908027984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f886406f5f0 con 0x7f886406f140 2024-09-10T20:59:50.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.447+0000 7f885a7fc640 1 -- 172.21.15.98:0/3908027984 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8860010860 con 0x7f886406f140 2024-09-10T20:59:50.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.447+0000 7f885a7fc640 1 --2- 172.21.15.98:0/3908027984 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8848077f70 0x7f884807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:50.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.448+0000 7f885a7fc640 1 -- 172.21.15.98:0/3908027984 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f886009a390 con 0x7f886406f140 2024-09-10T20:59:50.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.448+0000 7f88696b1640 1 --2- 172.21.15.98:0/3908027984 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8848077f70 0x7f884807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:50.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.448+0000 7f88696b1640 1 --2- 172.21.15.98:0/3908027984 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8848077f70 0x7f884807a430 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f88640b4f00 tx=0x7f8854009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:50.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.452+0000 7f885a7fc640 1 -- 172.21.15.98:0/3908027984 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f88600625a0 con 0x7f886406f140 2024-09-10T20:59:50.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.623+0000 7f886c116640 1 -- 172.21.15.98:0/3908027984 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f88640cd6d0 con 0x7f8848077f70 2024-09-10T20:59:50.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.627+0000 7f885a7fc640 1 -- 172.21.15.98:0/3908027984 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+238167 (secure 0 0 0) 0x7f88640cd6d0 con 0x7f8848077f70 2024-09-10T20:59:50.633 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:50.634 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-10T20:59:50.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.630+0000 7f886c116640 1 -- 172.21.15.98:0/3908027984 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8848077f70 msgr2=0x7f884807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.630+0000 7f886c116640 1 --2- 172.21.15.98:0/3908027984 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8848077f70 0x7f884807a430 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f88640b4f00 tx=0x7f8854009040 comp rx=0 tx=0).stop 2024-09-10T20:59:50.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.630+0000 7f886c116640 1 -- 172.21.15.98:0/3908027984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f886406f140 msgr2=0x7f88641f5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.630+0000 7f886c116640 1 --2- 172.21.15.98:0/3908027984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f886406f140 0x7f88641f5460 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f886000d950 tx=0x7f886000de10 comp rx=0 tx=0).stop 2024-09-10T20:59:50.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.630+0000 7f886c116640 1 -- 172.21.15.98:0/3908027984 shutdown_connections 2024-09-10T20:59:50.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.630+0000 7f886c116640 1 --2- 172.21.15.98:0/3908027984 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8848077f70 0x7f884807a430 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.630+0000 7f886c116640 1 --2- 172.21.15.98:0/3908027984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f886415c470 0x7f8864143880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.630+0000 7f886c116640 1 --2- 172.21.15.98:0/3908027984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8864073780 0x7f8864143340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.630+0000 7f886c116640 1 --2- 172.21.15.98:0/3908027984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f886406f140 0x7f88641f5460 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.630+0000 7f886c116640 1 -- 172.21.15.98:0/3908027984 >> 172.21.15.98:0/3908027984 conn(0x7f88640b1760 msgr2=0x7f88640b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:50.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.630+0000 7f886c116640 1 -- 172.21.15.98:0/3908027984 shutdown_connections 2024-09-10T20:59:50.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.630+0000 7f886c116640 1 -- 172.21.15.98:0/3908027984 wait complete. 2024-09-10T20:59:50.645 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-10T20:59:48.921015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":24,"acting":24,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":220604,"kb_used_data":4364,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754116,"statfs":{"total":773094113280,"available":772868214784,"internally_reserved":0,"allocated":4468736,"data_stored":2569899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000911"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.997501+0000","last_change":"2024-09-10T20:59:44.997501+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.997501+0000","last_undegraded":"2024-09-10T20:59:44.997501+0000","last_fullsized":"2024-09-10T20:59:44.997501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.995186+0000","last_change":"2024-09-10T20:59:44.995186+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.995186+0000","last_undegraded":"2024-09-10T20:59:44.995186+0000","last_fullsized":"2024-09-10T20:59:44.995186+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.999305+0000","last_change":"2024-09-10T20:59:44.999305+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.999305+0000","last_undegraded":"2024-09-10T20:59:44.999305+0000","last_fullsized":"2024-09-10T20:59:44.999305+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.994818+0000","last_change":"2024-09-10T20:59:44.994818+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.994818+0000","last_undegraded":"2024-09-10T20:59:44.994818+0000","last_fullsized":"2024-09-10T20:59:44.994818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.997501+0000","last_change":"2024-09-10T20:59:44.997501+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.997501+0000","last_undegraded":"2024-09-10T20:59:44.997501+0000","last_fullsized":"2024-09-10T20:59:44.997501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.998837+0000","last_change":"2024-09-10T20:59:44.998837+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.998837+0000","last_undegraded":"2024-09-10T20:59:44.998837+0000","last_fullsized":"2024-09-10T20:59:44.998837+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.995492+0000","last_change":"2024-09-10T20:59:44.995492+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.995492+0000","last_undegraded":"2024-09-10T20:59:44.995492+0000","last_fullsized":"2024-09-10T20:59:44.995492+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.998868+0000","last_change":"2024-09-10T20:59:44.998868+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.998868+0000","last_undegraded":"2024-09-10T20:59:44.998868+0000","last_fullsized":"2024-09-10T20:59:44.998868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.997504+0000","last_change":"2024-09-10T20:59:44.997504+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.997504+0000","last_undegraded":"2024-09-10T20:59:44.997504+0000","last_fullsized":"2024-09-10T20:59:44.997504+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.995527+0000","last_change":"2024-09-10T20:59:44.995527+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.995527+0000","last_undegraded":"2024-09-10T20:59:44.995527+0000","last_fullsized":"2024-09-10T20:59:44.995527+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:44.986452+0000","last_change":"2024-09-10T20:59:44.986452+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.986452+0000","last_undegraded":"2024-09-10T20:59:44.986452+0000","last_fullsized":"2024-09-10T20:59:44.986452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.998797+0000","last_change":"2024-09-10T20:59:44.998797+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.998797+0000","last_undegraded":"2024-09-10T20:59:44.998797+0000","last_fullsized":"2024-09-10T20:59:44.998797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":140,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T20:59:44.999128+0000","last_change":"2024-09-10T20:58:36.895993+0000","last_active":"2024-09-10T20:59:44.999128+0000","last_peered":"2024-09-10T20:59:44.999128+0000","last_clean":"2024-09-10T20:59:44.999128+0000","last_became_active":"2024-09-10T20:58:36.894988+0000","last_became_peered":"2024-09-10T20:58:36.894988+0000","last_unstale":"2024-09-10T20:59:44.999128+0000","last_undegraded":"2024-09-10T20:59:44.999128+0000","last_fullsized":"2024-09-10T20:59:44.999128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_clean_scrub_stamp":"2024-09-10T20:58:35.588702+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.082528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":22,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":767894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":177526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":177526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":171317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":171317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":171317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":999424,"data_stored":761685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":171317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:59:50.648 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:59:50.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.782+0000 7f9e95e5a640 1 -- 172.21.15.98:0/974764551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9006dcc0 msgr2=0x7f9e900b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.782+0000 7f9e95e5a640 1 --2- 172.21.15.98:0/974764551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9006dcc0 0x7f9e900b1740 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9e80005fd0 tx=0x7f9e8002a4b0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.782+0000 7f9e95e5a640 1 -- 172.21.15.98:0/974764551 shutdown_connections 2024-09-10T20:59:50.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.782+0000 7f9e95e5a640 1 --2- 172.21.15.98:0/974764551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9006dcc0 0x7f9e900b1740 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.782+0000 7f9e95e5a640 1 -- 172.21.15.98:0/974764551 >> 172.21.15.98:0/974764551 conn(0x7f9e90074150 msgr2=0x7f9e90074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:50.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.782+0000 7f9e95e5a640 1 -- 172.21.15.98:0/974764551 shutdown_connections 2024-09-10T20:59:50.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.782+0000 7f9e95e5a640 1 -- 172.21.15.98:0/974764551 wait complete. 2024-09-10T20:59:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.784+0000 7f9e95e5a640 1 Processor -- start 2024-09-10T20:59:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.784+0000 7f9e95e5a640 1 -- start start 2024-09-10T20:59:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.784+0000 7f9e95e5a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e9006cb70 0x7f9e9015db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.784+0000 7f9e95e5a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e9006dcc0 0x7f9e900994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.784+0000 7f9e95e5a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9006e0a0 0x7f9e90099a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.784+0000 7f9e95e5a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9e900b61a0 con 0x7f9e9006e0a0 2024-09-10T20:59:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.784+0000 7f9e95e5a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9e900b6020 con 0x7f9e9006cb70 2024-09-10T20:59:50.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.784+0000 7f9e95e5a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9e901661b0 con 0x7f9e9006dcc0 2024-09-10T20:59:50.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e8ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9006e0a0 0x7f9e90099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:50.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e8ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9006e0a0 0x7f9e90099a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54832/0 (socket says 172.21.15.98:54832) 2024-09-10T20:59:50.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e8ffff640 1 -- 172.21.15.98:0/2242353866 learned_addr learned my addr 172.21.15.98:0/2242353866 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:50.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e8ffff640 1 -- 172.21.15.98:0/2242353866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e9006dcc0 msgr2=0x7f9e900994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e8ffff640 1 --2- 172.21.15.98:0/2242353866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e9006dcc0 0x7f9e900994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e8ffff640 1 -- 172.21.15.98:0/2242353866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e9006cb70 msgr2=0x7f9e9015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e8f7fe640 1 --2- 172.21.15.98:0/2242353866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e9006cb70 0x7f9e9015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:50.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e8ffff640 1 --2- 172.21.15.98:0/2242353866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e9006cb70 0x7f9e9015db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e8ffff640 1 -- 172.21.15.98:0/2242353866 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e901633c0 con 0x7f9e9006e0a0 2024-09-10T20:59:50.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e8ffff640 1 --2- 172.21.15.98:0/2242353866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9006e0a0 0x7f9e90099a10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800c7b0 tx=0x7f9e8800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:50.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e77fff640 1 -- 172.21.15.98:0/2242353866 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88004b60 con 0x7f9e9006e0a0 2024-09-10T20:59:50.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e77fff640 1 -- 172.21.15.98:0/2242353866 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e88004df0 con 0x7f9e9006e0a0 2024-09-10T20:59:50.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e95e5a640 1 -- 172.21.15.98:0/2242353866 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e901636a0 con 0x7f9e9006e0a0 2024-09-10T20:59:50.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e77fff640 1 -- 172.21.15.98:0/2242353866 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8800aa90 con 0x7f9e9006e0a0 2024-09-10T20:59:50.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.785+0000 7f9e95e5a640 1 -- 172.21.15.98:0/2242353866 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e9015e0c0 con 0x7f9e9006e0a0 2024-09-10T20:59:50.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.786+0000 7f9e95e5a640 1 -- 172.21.15.98:0/2242353866 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e900b1740 con 0x7f9e9006e0a0 2024-09-10T20:59:50.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.787+0000 7f9e77fff640 1 -- 172.21.15.98:0/2242353866 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9e8800ac30 con 0x7f9e9006e0a0 2024-09-10T20:59:50.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.787+0000 7f9e77fff640 1 --2- 172.21.15.98:0/2242353866 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9e7c077f70 0x7f9e7c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:50.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.787+0000 7f9e77fff640 1 -- 172.21.15.98:0/2242353866 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f9e8809a130 con 0x7f9e9006e0a0 2024-09-10T20:59:50.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.788+0000 7f9e8f7fe640 1 --2- 172.21.15.98:0/2242353866 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9e7c077f70 0x7f9e7c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:50.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.788+0000 7f9e8f7fe640 1 --2- 172.21.15.98:0/2242353866 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9e7c077f70 0x7f9e7c07a430 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e80033000 tx=0x7f9e80004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:50.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.791+0000 7f9e77fff640 1 -- 172.21.15.98:0/2242353866 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9e88020070 con 0x7f9e9006e0a0 2024-09-10T20:59:50.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.962+0000 7f9e95e5a640 1 -- 172.21.15.98:0/2242353866 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e900d0ed0 con 0x7f9e7c077f70 2024-09-10T20:59:50.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.967+0000 7f9e77fff640 1 -- 172.21.15.98:0/2242353866 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247672 (secure 0 0 0) 0x7f9e900d0ed0 con 0x7f9e7c077f70 2024-09-10T20:59:50.973 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:50.974 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-10T20:59:50.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.970+0000 7f9e95e5a640 1 -- 172.21.15.98:0/2242353866 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9e7c077f70 msgr2=0x7f9e7c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.970+0000 7f9e95e5a640 1 --2- 172.21.15.98:0/2242353866 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9e7c077f70 0x7f9e7c07a430 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e80033000 tx=0x7f9e80004730 comp rx=0 tx=0).stop 2024-09-10T20:59:50.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.970+0000 7f9e95e5a640 1 -- 172.21.15.98:0/2242353866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9006e0a0 msgr2=0x7f9e90099a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.970+0000 7f9e95e5a640 1 --2- 172.21.15.98:0/2242353866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9006e0a0 0x7f9e90099a10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800c7b0 tx=0x7f9e8800cc70 comp rx=0 tx=0).stop 2024-09-10T20:59:50.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.970+0000 7f9e95e5a640 1 -- 172.21.15.98:0/2242353866 shutdown_connections 2024-09-10T20:59:50.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.970+0000 7f9e95e5a640 1 --2- 172.21.15.98:0/2242353866 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9e7c077f70 0x7f9e7c07a430 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.970+0000 7f9e95e5a640 1 --2- 172.21.15.98:0/2242353866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9006e0a0 0x7f9e90099a10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.970+0000 7f9e95e5a640 1 --2- 172.21.15.98:0/2242353866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e9006dcc0 0x7f9e900994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.970+0000 7f9e95e5a640 1 --2- 172.21.15.98:0/2242353866 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e9006cb70 0x7f9e9015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.970+0000 7f9e95e5a640 1 -- 172.21.15.98:0/2242353866 >> 172.21.15.98:0/2242353866 conn(0x7f9e90074150 msgr2=0x7f9e90159e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:50.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.970+0000 7f9e95e5a640 1 -- 172.21.15.98:0/2242353866 shutdown_connections 2024-09-10T20:59:50.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:50.970+0000 7f9e95e5a640 1 -- 172.21.15.98:0/2242353866 wait complete. 2024-09-10T20:59:50.984 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-09-10T20:59:50.921570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":124,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94,"ondisk_log_size":94,"up":148,"acting":148,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220872,"kb_used_data":4640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753848,"statfs":{"total":773094113280,"available":772867940352,"internally_reserved":0,"allocated":4751360,"data_stored":2705526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.928858"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008495+0000","last_change":"2024-09-10T20:59:47.020403+0000","last_active":"2024-09-10T20:59:48.008495+0000","last_peered":"2024-09-10T20:59:48.008495+0000","last_clean":"2024-09-10T20:59:48.008495+0000","last_became_active":"2024-09-10T20:59:47.018947+0000","last_became_peered":"2024-09-10T20:59:47.018947+0000","last_unstale":"2024-09-10T20:59:48.008495+0000","last_undegraded":"2024-09-10T20:59:48.008495+0000","last_fullsized":"2024-09-10T20:59:48.008495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:41.452362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008383+0000","last_change":"2024-09-10T20:59:47.022856+0000","last_active":"2024-09-10T20:59:48.008383+0000","last_peered":"2024-09-10T20:59:48.008383+0000","last_clean":"2024-09-10T20:59:48.008383+0000","last_became_active":"2024-09-10T20:59:47.022579+0000","last_became_peered":"2024-09-10T20:59:47.022579+0000","last_unstale":"2024-09-10T20:59:48.008383+0000","last_undegraded":"2024-09-10T20:59:48.008383+0000","last_fullsized":"2024-09-10T20:59:48.008383+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:51.434814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371766+0000","last_change":"2024-09-10T20:59:47.028091+0000","last_active":"2024-09-10T20:59:48.371766+0000","last_peered":"2024-09-10T20:59:48.371766+0000","last_clean":"2024-09-10T20:59:48.371766+0000","last_became_active":"2024-09-10T20:59:47.027830+0000","last_became_peered":"2024-09-10T20:59:47.027830+0000","last_unstale":"2024-09-10T20:59:48.371766+0000","last_undegraded":"2024-09-10T20:59:48.371766+0000","last_fullsized":"2024-09-10T20:59:48.371766+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:17.875896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024599+0000","last_change":"2024-09-10T20:59:47.033657+0000","last_active":"2024-09-10T20:59:48.024599+0000","last_peered":"2024-09-10T20:59:48.024599+0000","last_clean":"2024-09-10T20:59:48.024599+0000","last_became_active":"2024-09-10T20:59:47.033363+0000","last_became_peered":"2024-09-10T20:59:47.033363+0000","last_unstale":"2024-09-10T20:59:48.024599+0000","last_undegraded":"2024-09-10T20:59:48.024599+0000","last_fullsized":"2024-09-10T20:59:48.024599+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:45.439857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024571+0000","last_change":"2024-09-10T20:59:47.030433+0000","last_active":"2024-09-10T20:59:48.024571+0000","last_peered":"2024-09-10T20:59:48.024571+0000","last_clean":"2024-09-10T20:59:48.024571+0000","last_became_active":"2024-09-10T20:59:47.030093+0000","last_became_peered":"2024-09-10T20:59:47.030093+0000","last_unstale":"2024-09-10T20:59:48.024571+0000","last_undegraded":"2024-09-10T20:59:48.024571+0000","last_fullsized":"2024-09-10T20:59:48.024571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:23.601594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371305+0000","last_change":"2024-09-10T20:59:47.022075+0000","last_active":"2024-09-10T20:59:48.371305+0000","last_peered":"2024-09-10T20:59:48.371305+0000","last_clean":"2024-09-10T20:59:48.371305+0000","last_became_active":"2024-09-10T20:59:47.021810+0000","last_became_peered":"2024-09-10T20:59:47.021810+0000","last_unstale":"2024-09-10T20:59:48.371305+0000","last_undegraded":"2024-09-10T20:59:48.371305+0000","last_fullsized":"2024-09-10T20:59:48.371305+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:53.763319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009198+0000","last_change":"2024-09-10T20:59:47.024763+0000","last_active":"2024-09-10T20:59:48.009198+0000","last_peered":"2024-09-10T20:59:48.009198+0000","last_clean":"2024-09-10T20:59:48.009198+0000","last_became_active":"2024-09-10T20:59:47.024475+0000","last_became_peered":"2024-09-10T20:59:47.024475+0000","last_unstale":"2024-09-10T20:59:48.009198+0000","last_undegraded":"2024-09-10T20:59:48.009198+0000","last_fullsized":"2024-09-10T20:59:48.009198+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:51:57.920889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008071+0000","last_change":"2024-09-10T20:59:47.026033+0000","last_active":"2024-09-10T20:59:48.008071+0000","last_peered":"2024-09-10T20:59:48.008071+0000","last_clean":"2024-09-10T20:59:48.008071+0000","last_became_active":"2024-09-10T20:59:47.025434+0000","last_became_peered":"2024-09-10T20:59:47.025434+0000","last_unstale":"2024-09-10T20:59:48.008071+0000","last_undegraded":"2024-09-10T20:59:48.008071+0000","last_fullsized":"2024-09-10T20:59:48.008071+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:28.753205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371967+0000","last_change":"2024-09-10T20:59:47.028797+0000","last_active":"2024-09-10T20:59:48.371967+0000","last_peered":"2024-09-10T20:59:48.371967+0000","last_clean":"2024-09-10T20:59:48.371967+0000","last_became_active":"2024-09-10T20:59:47.028522+0000","last_became_peered":"2024-09-10T20:59:47.028522+0000","last_unstale":"2024-09-10T20:59:48.371967+0000","last_undegraded":"2024-09-10T20:59:48.371967+0000","last_fullsized":"2024-09-10T20:59:48.371967+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:35.047921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010611+0000","last_change":"2024-09-10T20:59:47.028365+0000","last_active":"2024-09-10T20:59:48.010611+0000","last_peered":"2024-09-10T20:59:48.010611+0000","last_clean":"2024-09-10T20:59:48.010611+0000","last_became_active":"2024-09-10T20:59:47.028064+0000","last_became_peered":"2024-09-10T20:59:47.028064+0000","last_unstale":"2024-09-10T20:59:48.010611+0000","last_undegraded":"2024-09-10T20:59:48.010611+0000","last_fullsized":"2024-09-10T20:59:48.010611+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:39.938414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009375+0000","last_change":"2024-09-10T20:59:47.031189+0000","last_active":"2024-09-10T20:59:48.009375+0000","last_peered":"2024-09-10T20:59:48.009375+0000","last_clean":"2024-09-10T20:59:48.009375+0000","last_became_active":"2024-09-10T20:59:47.030852+0000","last_became_peered":"2024-09-10T20:59:47.030852+0000","last_unstale":"2024-09-10T20:59:48.009375+0000","last_undegraded":"2024-09-10T20:59:48.009375+0000","last_fullsized":"2024-09-10T20:59:48.009375+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:49.659631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371334+0000","last_change":"2024-09-10T20:59:47.024031+0000","last_active":"2024-09-10T20:59:48.371334+0000","last_peered":"2024-09-10T20:59:48.371334+0000","last_clean":"2024-09-10T20:59:48.371334+0000","last_became_active":"2024-09-10T20:59:47.023695+0000","last_became_peered":"2024-09-10T20:59:47.023695+0000","last_unstale":"2024-09-10T20:59:48.371334+0000","last_undegraded":"2024-09-10T20:59:48.371334+0000","last_fullsized":"2024-09-10T20:59:48.371334+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:09.439607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009786+0000","last_change":"2024-09-10T20:59:47.028762+0000","last_active":"2024-09-10T20:59:48.009786+0000","last_peered":"2024-09-10T20:59:48.009786+0000","last_clean":"2024-09-10T20:59:48.009786+0000","last_became_active":"2024-09-10T20:59:47.028406+0000","last_became_peered":"2024-09-10T20:59:47.028406+0000","last_unstale":"2024-09-10T20:59:48.009786+0000","last_undegraded":"2024-09-10T20:59:48.009786+0000","last_fullsized":"2024-09-10T20:59:48.009786+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:12.795591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009388+0000","last_change":"2024-09-10T20:59:47.024046+0000","last_active":"2024-09-10T20:59:48.009388+0000","last_peered":"2024-09-10T20:59:48.009388+0000","last_clean":"2024-09-10T20:59:48.009388+0000","last_became_active":"2024-09-10T20:59:47.023745+0000","last_became_peered":"2024-09-10T20:59:47.023745+0000","last_unstale":"2024-09-10T20:59:48.009388+0000","last_undegraded":"2024-09-10T20:59:48.009388+0000","last_fullsized":"2024-09-10T20:59:48.009388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:32.861565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024695+0000","last_change":"2024-09-10T20:59:47.030399+0000","last_active":"2024-09-10T20:59:48.024695+0000","last_peered":"2024-09-10T20:59:48.024695+0000","last_clean":"2024-09-10T20:59:48.024695+0000","last_became_active":"2024-09-10T20:59:47.029699+0000","last_became_peered":"2024-09-10T20:59:47.029699+0000","last_unstale":"2024-09-10T20:59:48.024695+0000","last_undegraded":"2024-09-10T20:59:48.024695+0000","last_fullsized":"2024-09-10T20:59:48.024695+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:26.745564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009982+0000","last_change":"2024-09-10T20:59:47.028032+0000","last_active":"2024-09-10T20:59:48.009982+0000","last_peered":"2024-09-10T20:59:48.009982+0000","last_clean":"2024-09-10T20:59:48.009982+0000","last_became_active":"2024-09-10T20:59:47.027708+0000","last_became_peered":"2024-09-10T20:59:47.027708+0000","last_unstale":"2024-09-10T20:59:48.009982+0000","last_undegraded":"2024-09-10T20:59:48.009982+0000","last_fullsized":"2024-09-10T20:59:48.009982+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:48.185509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024474+0000","last_change":"2024-09-10T20:59:47.027243+0000","last_active":"2024-09-10T20:59:48.024474+0000","last_peered":"2024-09-10T20:59:48.024474+0000","last_clean":"2024-09-10T20:59:48.024474+0000","last_became_active":"2024-09-10T20:59:47.026976+0000","last_became_peered":"2024-09-10T20:59:47.026976+0000","last_unstale":"2024-09-10T20:59:48.024474+0000","last_undegraded":"2024-09-10T20:59:48.024474+0000","last_fullsized":"2024-09-10T20:59:48.024474+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:17.556094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371446+0000","last_change":"2024-09-10T20:59:47.029424+0000","last_active":"2024-09-10T20:59:48.371446+0000","last_peered":"2024-09-10T20:59:48.371446+0000","last_clean":"2024-09-10T20:59:48.371446+0000","last_became_active":"2024-09-10T20:59:47.029180+0000","last_became_peered":"2024-09-10T20:59:47.029180+0000","last_unstale":"2024-09-10T20:59:48.371446+0000","last_undegraded":"2024-09-10T20:59:48.371446+0000","last_fullsized":"2024-09-10T20:59:48.371446+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:31.373458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010216+0000","last_change":"2024-09-10T20:59:47.025305+0000","last_active":"2024-09-10T20:59:48.010216+0000","last_peered":"2024-09-10T20:59:48.010216+0000","last_clean":"2024-09-10T20:59:48.010216+0000","last_became_active":"2024-09-10T20:59:47.025036+0000","last_became_peered":"2024-09-10T20:59:47.025036+0000","last_unstale":"2024-09-10T20:59:48.010216+0000","last_undegraded":"2024-09-10T20:59:48.010216+0000","last_fullsized":"2024-09-10T20:59:48.010216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:25.337419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371305+0000","last_change":"2024-09-10T20:59:47.021237+0000","last_active":"2024-09-10T20:59:48.371305+0000","last_peered":"2024-09-10T20:59:48.371305+0000","last_clean":"2024-09-10T20:59:48.371305+0000","last_became_active":"2024-09-10T20:59:47.020964+0000","last_became_peered":"2024-09-10T20:59:47.020964+0000","last_unstale":"2024-09-10T20:59:48.371305+0000","last_undegraded":"2024-09-10T20:59:48.371305+0000","last_fullsized":"2024-09-10T20:59:48.371305+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:50.897033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.997501+0000","last_change":"2024-09-10T20:59:44.997501+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.997501+0000","last_undegraded":"2024-09-10T20:59:44.997501+0000","last_fullsized":"2024-09-10T20:59:44.997501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371836+0000","last_change":"2024-09-10T20:59:47.028678+0000","last_active":"2024-09-10T20:59:48.371836+0000","last_peered":"2024-09-10T20:59:48.371836+0000","last_clean":"2024-09-10T20:59:48.371836+0000","last_became_active":"2024-09-10T20:59:47.028134+0000","last_became_peered":"2024-09-10T20:59:47.028134+0000","last_unstale":"2024-09-10T20:59:48.371836+0000","last_undegraded":"2024-09-10T20:59:48.371836+0000","last_fullsized":"2024-09-10T20:59:48.371836+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:21.297545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371469+0000","last_change":"2024-09-10T20:59:46.014029+0000","last_active":"2024-09-10T20:59:48.371469+0000","last_peered":"2024-09-10T20:59:48.371469+0000","last_clean":"2024-09-10T20:59:48.371469+0000","last_became_active":"2024-09-10T20:59:46.013771+0000","last_became_peered":"2024-09-10T20:59:46.013771+0000","last_unstale":"2024-09-10T20:59:48.371469+0000","last_undegraded":"2024-09-10T20:59:48.371469+0000","last_fullsized":"2024-09-10T20:59:48.371469+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:05.365319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371865+0000","last_change":"2024-09-10T20:59:46.012293+0000","last_active":"2024-09-10T20:59:48.371865+0000","last_peered":"2024-09-10T20:59:48.371865+0000","last_clean":"2024-09-10T20:59:48.371865+0000","last_became_active":"2024-09-10T20:59:46.012014+0000","last_became_peered":"2024-09-10T20:59:46.012014+0000","last_unstale":"2024-09-10T20:59:48.371865+0000","last_undegraded":"2024-09-10T20:59:48.371865+0000","last_fullsized":"2024-09-10T20:59:48.371865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:30.723075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008466+0000","last_change":"2024-09-10T20:59:47.032715+0000","last_active":"2024-09-10T20:59:48.008466+0000","last_peered":"2024-09-10T20:59:48.008466+0000","last_clean":"2024-09-10T20:59:48.008466+0000","last_became_active":"2024-09-10T20:59:47.032448+0000","last_became_peered":"2024-09-10T20:59:47.032448+0000","last_unstale":"2024-09-10T20:59:48.008466+0000","last_undegraded":"2024-09-10T20:59:48.008466+0000","last_fullsized":"2024-09-10T20:59:48.008466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:34.824871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.995186+0000","last_change":"2024-09-10T20:59:44.995186+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.995186+0000","last_undegraded":"2024-09-10T20:59:44.995186+0000","last_fullsized":"2024-09-10T20:59:44.995186+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371308+0000","last_change":"2024-09-10T20:59:47.026327+0000","last_active":"2024-09-10T20:59:48.371308+0000","last_peered":"2024-09-10T20:59:48.371308+0000","last_clean":"2024-09-10T20:59:48.371308+0000","last_became_active":"2024-09-10T20:59:47.026074+0000","last_became_peered":"2024-09-10T20:59:47.026074+0000","last_unstale":"2024-09-10T20:59:48.371308+0000","last_undegraded":"2024-09-10T20:59:48.371308+0000","last_fullsized":"2024-09-10T20:59:48.371308+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:41.166136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010096+0000","last_change":"2024-09-10T20:59:46.014169+0000","last_active":"2024-09-10T20:59:48.010096+0000","last_peered":"2024-09-10T20:59:48.010096+0000","last_clean":"2024-09-10T20:59:48.010096+0000","last_became_active":"2024-09-10T20:59:46.013940+0000","last_became_peered":"2024-09-10T20:59:46.013940+0000","last_unstale":"2024-09-10T20:59:48.010096+0000","last_undegraded":"2024-09-10T20:59:48.010096+0000","last_fullsized":"2024-09-10T20:59:48.010096+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:52.758336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009576+0000","last_change":"2024-09-10T20:59:47.027620+0000","last_active":"2024-09-10T20:59:48.009576+0000","last_peered":"2024-09-10T20:59:48.009576+0000","last_clean":"2024-09-10T20:59:48.009576+0000","last_became_active":"2024-09-10T20:59:47.027339+0000","last_became_peered":"2024-09-10T20:59:47.027339+0000","last_unstale":"2024-09-10T20:59:48.009576+0000","last_undegraded":"2024-09-10T20:59:48.009576+0000","last_fullsized":"2024-09-10T20:59:48.009576+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:37.031813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.999305+0000","last_change":"2024-09-10T20:59:44.999305+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.999305+0000","last_undegraded":"2024-09-10T20:59:44.999305+0000","last_fullsized":"2024-09-10T20:59:44.999305+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371718+0000","last_change":"2024-09-10T20:59:46.012513+0000","last_active":"2024-09-10T20:59:48.371718+0000","last_peered":"2024-09-10T20:59:48.371718+0000","last_clean":"2024-09-10T20:59:48.371718+0000","last_became_active":"2024-09-10T20:59:46.012251+0000","last_became_peered":"2024-09-10T20:59:46.012251+0000","last_unstale":"2024-09-10T20:59:48.371718+0000","last_undegraded":"2024-09-10T20:59:48.371718+0000","last_fullsized":"2024-09-10T20:59:48.371718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:06:09.764168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008181+0000","last_change":"2024-09-10T20:59:47.027750+0000","last_active":"2024-09-10T20:59:48.008181+0000","last_peered":"2024-09-10T20:59:48.008181+0000","last_clean":"2024-09-10T20:59:48.008181+0000","last_became_active":"2024-09-10T20:59:47.027454+0000","last_became_peered":"2024-09-10T20:59:47.027454+0000","last_unstale":"2024-09-10T20:59:48.008181+0000","last_undegraded":"2024-09-10T20:59:48.008181+0000","last_fullsized":"2024-09-10T20:59:48.008181+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:48.206623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.994818+0000","last_change":"2024-09-10T20:59:44.994818+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.994818+0000","last_undegraded":"2024-09-10T20:59:44.994818+0000","last_fullsized":"2024-09-10T20:59:44.994818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024638+0000","last_change":"2024-09-10T20:59:47.030452+0000","last_active":"2024-09-10T20:59:48.024638+0000","last_peered":"2024-09-10T20:59:48.024638+0000","last_clean":"2024-09-10T20:59:48.024638+0000","last_became_active":"2024-09-10T20:59:47.030133+0000","last_became_peered":"2024-09-10T20:59:47.030133+0000","last_unstale":"2024-09-10T20:59:48.024638+0000","last_undegraded":"2024-09-10T20:59:48.024638+0000","last_fullsized":"2024-09-10T20:59:48.024638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:56.952358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008556+0000","last_change":"2024-09-10T20:59:46.014818+0000","last_active":"2024-09-10T20:59:48.008556+0000","last_peered":"2024-09-10T20:59:48.008556+0000","last_clean":"2024-09-10T20:59:48.008556+0000","last_became_active":"2024-09-10T20:59:46.014578+0000","last_became_peered":"2024-09-10T20:59:46.014578+0000","last_unstale":"2024-09-10T20:59:48.008556+0000","last_undegraded":"2024-09-10T20:59:48.008556+0000","last_fullsized":"2024-09-10T20:59:48.008556+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:30.078404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010561+0000","last_change":"2024-09-10T20:59:46.007567+0000","last_active":"2024-09-10T20:59:48.010561+0000","last_peered":"2024-09-10T20:59:48.010561+0000","last_clean":"2024-09-10T20:59:48.010561+0000","last_became_active":"2024-09-10T20:59:46.006773+0000","last_became_peered":"2024-09-10T20:59:46.006773+0000","last_unstale":"2024-09-10T20:59:48.010561+0000","last_undegraded":"2024-09-10T20:59:48.010561+0000","last_fullsized":"2024-09-10T20:59:48.010561+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:19.491620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371345+0000","last_change":"2024-09-10T20:59:47.027217+0000","last_active":"2024-09-10T20:59:48.371345+0000","last_peered":"2024-09-10T20:59:48.371345+0000","last_clean":"2024-09-10T20:59:48.371345+0000","last_became_active":"2024-09-10T20:59:47.026939+0000","last_became_peered":"2024-09-10T20:59:47.026939+0000","last_unstale":"2024-09-10T20:59:48.371345+0000","last_undegraded":"2024-09-10T20:59:48.371345+0000","last_fullsized":"2024-09-10T20:59:48.371345+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:25.592848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009689+0000","last_change":"2024-09-10T20:59:46.010441+0000","last_active":"2024-09-10T20:59:48.009689+0000","last_peered":"2024-09-10T20:59:48.009689+0000","last_clean":"2024-09-10T20:59:48.009689+0000","last_became_active":"2024-09-10T20:59:46.010052+0000","last_became_peered":"2024-09-10T20:59:46.010052+0000","last_unstale":"2024-09-10T20:59:48.009689+0000","last_undegraded":"2024-09-10T20:59:48.009689+0000","last_fullsized":"2024-09-10T20:59:48.009689+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:19.191818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008249+0000","last_change":"2024-09-10T20:59:46.008318+0000","last_active":"2024-09-10T20:59:48.008249+0000","last_peered":"2024-09-10T20:59:48.008249+0000","last_clean":"2024-09-10T20:59:48.008249+0000","last_became_active":"2024-09-10T20:59:46.008069+0000","last_became_peered":"2024-09-10T20:59:46.008069+0000","last_unstale":"2024-09-10T20:59:48.008249+0000","last_undegraded":"2024-09-10T20:59:48.008249+0000","last_fullsized":"2024-09-10T20:59:48.008249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:51.723589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371531+0000","last_change":"2024-09-10T20:59:47.028617+0000","last_active":"2024-09-10T20:59:48.371531+0000","last_peered":"2024-09-10T20:59:48.371531+0000","last_clean":"2024-09-10T20:59:48.371531+0000","last_became_active":"2024-09-10T20:59:47.028282+0000","last_became_peered":"2024-09-10T20:59:47.028282+0000","last_unstale":"2024-09-10T20:59:48.371531+0000","last_undegraded":"2024-09-10T20:59:48.371531+0000","last_fullsized":"2024-09-10T20:59:48.371531+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:35.832692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008386+0000","last_change":"2024-09-10T20:59:46.016007+0000","last_active":"2024-09-10T20:59:48.008386+0000","last_peered":"2024-09-10T20:59:48.008386+0000","last_clean":"2024-09-10T20:59:48.008386+0000","last_became_active":"2024-09-10T20:59:46.015529+0000","last_became_peered":"2024-09-10T20:59:46.015529+0000","last_unstale":"2024-09-10T20:59:48.008386+0000","last_undegraded":"2024-09-10T20:59:48.008386+0000","last_fullsized":"2024-09-10T20:59:48.008386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:58.373425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008159+0000","last_change":"2024-09-10T20:59:47.018746+0000","last_active":"2024-09-10T20:59:48.008159+0000","last_peered":"2024-09-10T20:59:48.008159+0000","last_clean":"2024-09-10T20:59:48.008159+0000","last_became_active":"2024-09-10T20:59:47.018482+0000","last_became_peered":"2024-09-10T20:59:47.018482+0000","last_unstale":"2024-09-10T20:59:48.008159+0000","last_undegraded":"2024-09-10T20:59:48.008159+0000","last_fullsized":"2024-09-10T20:59:48.008159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:17.605108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371407+0000","last_change":"2024-09-10T20:59:46.013441+0000","last_active":"2024-09-10T20:59:48.371407+0000","last_peered":"2024-09-10T20:59:48.371407+0000","last_clean":"2024-09-10T20:59:48.371407+0000","last_became_active":"2024-09-10T20:59:46.012202+0000","last_became_peered":"2024-09-10T20:59:46.012202+0000","last_unstale":"2024-09-10T20:59:48.371407+0000","last_undegraded":"2024-09-10T20:59:48.371407+0000","last_fullsized":"2024-09-10T20:59:48.371407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:59.817251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.997501+0000","last_change":"2024-09-10T20:59:44.997501+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.997501+0000","last_undegraded":"2024-09-10T20:59:44.997501+0000","last_fullsized":"2024-09-10T20:59:44.997501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008204+0000","last_change":"2024-09-10T20:59:47.018576+0000","last_active":"2024-09-10T20:59:48.008204+0000","last_peered":"2024-09-10T20:59:48.008204+0000","last_clean":"2024-09-10T20:59:48.008204+0000","last_became_active":"2024-09-10T20:59:47.018285+0000","last_became_peered":"2024-09-10T20:59:47.018285+0000","last_unstale":"2024-09-10T20:59:48.008204+0000","last_undegraded":"2024-09-10T20:59:48.008204+0000","last_fullsized":"2024-09-10T20:59:48.008204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:32.917475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.998837+0000","last_change":"2024-09-10T20:59:44.998837+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.998837+0000","last_undegraded":"2024-09-10T20:59:44.998837+0000","last_fullsized":"2024-09-10T20:59:44.998837+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008333+0000","last_change":"2024-09-10T20:59:47.037762+0000","last_active":"2024-09-10T20:59:48.008333+0000","last_peered":"2024-09-10T20:59:48.008333+0000","last_clean":"2024-09-10T20:59:48.008333+0000","last_became_active":"2024-09-10T20:59:47.037007+0000","last_became_peered":"2024-09-10T20:59:47.037007+0000","last_unstale":"2024-09-10T20:59:48.008333+0000","last_undegraded":"2024-09-10T20:59:48.008333+0000","last_fullsized":"2024-09-10T20:59:48.008333+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:03.943638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010092+0000","last_change":"2024-09-10T20:59:46.009743+0000","last_active":"2024-09-10T20:59:48.010092+0000","last_peered":"2024-09-10T20:59:48.010092+0000","last_clean":"2024-09-10T20:59:48.010092+0000","last_became_active":"2024-09-10T20:59:46.009491+0000","last_became_peered":"2024-09-10T20:59:46.009491+0000","last_unstale":"2024-09-10T20:59:48.010092+0000","last_undegraded":"2024-09-10T20:59:48.010092+0000","last_fullsized":"2024-09-10T20:59:48.010092+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:56.031945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009586+0000","last_change":"2024-09-10T20:59:47.026805+0000","last_active":"2024-09-10T20:59:48.009586+0000","last_peered":"2024-09-10T20:59:48.009586+0000","last_clean":"2024-09-10T20:59:48.009586+0000","last_became_active":"2024-09-10T20:59:47.026545+0000","last_became_peered":"2024-09-10T20:59:47.026545+0000","last_unstale":"2024-09-10T20:59:48.009586+0000","last_undegraded":"2024-09-10T20:59:48.009586+0000","last_fullsized":"2024-09-10T20:59:48.009586+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:47.670364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.995492+0000","last_change":"2024-09-10T20:59:44.995492+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.995492+0000","last_undegraded":"2024-09-10T20:59:44.995492+0000","last_fullsized":"2024-09-10T20:59:44.995492+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024663+0000","last_change":"2024-09-10T20:59:46.010713+0000","last_active":"2024-09-10T20:59:48.024663+0000","last_peered":"2024-09-10T20:59:48.024663+0000","last_clean":"2024-09-10T20:59:48.024663+0000","last_became_active":"2024-09-10T20:59:46.010448+0000","last_became_peered":"2024-09-10T20:59:46.010448+0000","last_unstale":"2024-09-10T20:59:48.024663+0000","last_undegraded":"2024-09-10T20:59:48.024663+0000","last_fullsized":"2024-09-10T20:59:48.024663+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:41:37.633483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024766+0000","last_change":"2024-09-10T20:59:47.018725+0000","last_active":"2024-09-10T20:59:48.024766+0000","last_peered":"2024-09-10T20:59:48.024766+0000","last_clean":"2024-09-10T20:59:48.024766+0000","last_became_active":"2024-09-10T20:59:47.018486+0000","last_became_peered":"2024-09-10T20:59:47.018486+0000","last_unstale":"2024-09-10T20:59:48.024766+0000","last_undegraded":"2024-09-10T20:59:48.024766+0000","last_fullsized":"2024-09-10T20:59:48.024766+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:37.582285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371690+0000","last_change":"2024-09-10T20:59:46.013035+0000","last_active":"2024-09-10T20:59:48.371690+0000","last_peered":"2024-09-10T20:59:48.371690+0000","last_clean":"2024-09-10T20:59:48.371690+0000","last_became_active":"2024-09-10T20:59:46.012519+0000","last_became_peered":"2024-09-10T20:59:46.012519+0000","last_unstale":"2024-09-10T20:59:48.371690+0000","last_undegraded":"2024-09-10T20:59:48.371690+0000","last_fullsized":"2024-09-10T20:59:48.371690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:38.211184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024461+0000","last_change":"2024-09-10T20:59:46.015090+0000","last_active":"2024-09-10T20:59:48.024461+0000","last_peered":"2024-09-10T20:59:48.024461+0000","last_clean":"2024-09-10T20:59:48.024461+0000","last_became_active":"2024-09-10T20:59:46.014824+0000","last_became_peered":"2024-09-10T20:59:46.014824+0000","last_unstale":"2024-09-10T20:59:48.024461+0000","last_undegraded":"2024-09-10T20:59:48.024461+0000","last_fullsized":"2024-09-10T20:59:48.024461+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:44.918062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371633+0000","last_change":"2024-09-10T20:59:47.028376+0000","last_active":"2024-09-10T20:59:48.371633+0000","last_peered":"2024-09-10T20:59:48.371633+0000","last_clean":"2024-09-10T20:59:48.371633+0000","last_became_active":"2024-09-10T20:59:47.027880+0000","last_became_peered":"2024-09-10T20:59:47.027880+0000","last_unstale":"2024-09-10T20:59:48.371633+0000","last_undegraded":"2024-09-10T20:59:48.371633+0000","last_fullsized":"2024-09-10T20:59:48.371633+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:24.095188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024456+0000","last_change":"2024-09-10T20:59:46.011199+0000","last_active":"2024-09-10T20:59:48.024456+0000","last_peered":"2024-09-10T20:59:48.024456+0000","last_clean":"2024-09-10T20:59:48.024456+0000","last_became_active":"2024-09-10T20:59:46.010933+0000","last_became_peered":"2024-09-10T20:59:46.010933+0000","last_unstale":"2024-09-10T20:59:48.024456+0000","last_undegraded":"2024-09-10T20:59:48.024456+0000","last_fullsized":"2024-09-10T20:59:48.024456+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:37.482570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009795+0000","last_change":"2024-09-10T20:59:47.031076+0000","last_active":"2024-09-10T20:59:48.009795+0000","last_peered":"2024-09-10T20:59:48.009795+0000","last_clean":"2024-09-10T20:59:48.009795+0000","last_became_active":"2024-09-10T20:59:47.030753+0000","last_became_peered":"2024-09-10T20:59:47.030753+0000","last_unstale":"2024-09-10T20:59:48.009795+0000","last_undegraded":"2024-09-10T20:59:48.009795+0000","last_fullsized":"2024-09-10T20:59:48.009795+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:43.434063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.998868+0000","last_change":"2024-09-10T20:59:44.998868+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.998868+0000","last_undegraded":"2024-09-10T20:59:44.998868+0000","last_fullsized":"2024-09-10T20:59:44.998868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008402+0000","last_change":"2024-09-10T20:59:47.016933+0000","last_active":"2024-09-10T20:59:48.008402+0000","last_peered":"2024-09-10T20:59:48.008402+0000","last_clean":"2024-09-10T20:59:48.008402+0000","last_became_active":"2024-09-10T20:59:47.016668+0000","last_became_peered":"2024-09-10T20:59:47.016668+0000","last_unstale":"2024-09-10T20:59:48.008402+0000","last_undegraded":"2024-09-10T20:59:48.008402+0000","last_fullsized":"2024-09-10T20:59:48.008402+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:02.940429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024373+0000","last_change":"2024-09-10T20:59:46.015133+0000","last_active":"2024-09-10T20:59:48.024373+0000","last_peered":"2024-09-10T20:59:48.024373+0000","last_clean":"2024-09-10T20:59:48.024373+0000","last_became_active":"2024-09-10T20:59:46.014855+0000","last_became_peered":"2024-09-10T20:59:46.014855+0000","last_unstale":"2024-09-10T20:59:48.024373+0000","last_undegraded":"2024-09-10T20:59:48.024373+0000","last_fullsized":"2024-09-10T20:59:48.024373+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:16.570644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371608+0000","last_change":"2024-09-10T20:59:47.028767+0000","last_active":"2024-09-10T20:59:48.371608+0000","last_peered":"2024-09-10T20:59:48.371608+0000","last_clean":"2024-09-10T20:59:48.371608+0000","last_became_active":"2024-09-10T20:59:47.028529+0000","last_became_peered":"2024-09-10T20:59:47.028529+0000","last_unstale":"2024-09-10T20:59:48.371608+0000","last_undegraded":"2024-09-10T20:59:48.371608+0000","last_fullsized":"2024-09-10T20:59:48.371608+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:19.645203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010215+0000","last_change":"2024-09-10T20:59:46.006840+0000","last_active":"2024-09-10T20:59:48.010215+0000","last_peered":"2024-09-10T20:59:48.010215+0000","last_clean":"2024-09-10T20:59:48.010215+0000","last_became_active":"2024-09-10T20:59:46.006570+0000","last_became_peered":"2024-09-10T20:59:46.006570+0000","last_unstale":"2024-09-10T20:59:48.010215+0000","last_undegraded":"2024-09-10T20:59:48.010215+0000","last_fullsized":"2024-09-10T20:59:48.010215+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:11.450731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.026242+0000","last_change":"2024-09-10T20:59:46.013179+0000","last_active":"2024-09-10T20:59:48.026242+0000","last_peered":"2024-09-10T20:59:48.026242+0000","last_clean":"2024-09-10T20:59:48.026242+0000","last_became_active":"2024-09-10T20:59:46.012907+0000","last_became_peered":"2024-09-10T20:59:46.012907+0000","last_unstale":"2024-09-10T20:59:48.026242+0000","last_undegraded":"2024-09-10T20:59:48.026242+0000","last_fullsized":"2024-09-10T20:59:48.026242+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:19.289722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008396+0000","last_change":"2024-09-10T20:59:47.034104+0000","last_active":"2024-09-10T20:59:48.008396+0000","last_peered":"2024-09-10T20:59:48.008396+0000","last_clean":"2024-09-10T20:59:48.008396+0000","last_became_active":"2024-09-10T20:59:47.033795+0000","last_became_peered":"2024-09-10T20:59:47.033795+0000","last_unstale":"2024-09-10T20:59:48.008396+0000","last_undegraded":"2024-09-10T20:59:48.008396+0000","last_fullsized":"2024-09-10T20:59:48.008396+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:59.999686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371561+0000","last_change":"2024-09-10T20:59:46.017352+0000","last_active":"2024-09-10T20:59:48.371561+0000","last_peered":"2024-09-10T20:59:48.371561+0000","last_clean":"2024-09-10T20:59:48.371561+0000","last_became_active":"2024-09-10T20:59:46.017073+0000","last_became_peered":"2024-09-10T20:59:46.017073+0000","last_unstale":"2024-09-10T20:59:48.371561+0000","last_undegraded":"2024-09-10T20:59:48.371561+0000","last_fullsized":"2024-09-10T20:59:48.371561+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:37.017258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371532+0000","last_change":"2024-09-10T20:59:47.025517+0000","last_active":"2024-09-10T20:59:48.371532+0000","last_peered":"2024-09-10T20:59:48.371532+0000","last_clean":"2024-09-10T20:59:48.371532+0000","last_became_active":"2024-09-10T20:59:47.025246+0000","last_became_peered":"2024-09-10T20:59:47.025246+0000","last_unstale":"2024-09-10T20:59:48.371532+0000","last_undegraded":"2024-09-10T20:59:48.371532+0000","last_fullsized":"2024-09-10T20:59:48.371532+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:32.064402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371860+0000","last_change":"2024-09-10T20:59:46.015703+0000","last_active":"2024-09-10T20:59:48.371860+0000","last_peered":"2024-09-10T20:59:48.371860+0000","last_clean":"2024-09-10T20:59:48.371860+0000","last_became_active":"2024-09-10T20:59:46.015257+0000","last_became_peered":"2024-09-10T20:59:46.015257+0000","last_unstale":"2024-09-10T20:59:48.371860+0000","last_undegraded":"2024-09-10T20:59:48.371860+0000","last_fullsized":"2024-09-10T20:59:48.371860+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:24:41.884818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371619+0000","last_change":"2024-09-10T20:59:47.031083+0000","last_active":"2024-09-10T20:59:48.371619+0000","last_peered":"2024-09-10T20:59:48.371619+0000","last_clean":"2024-09-10T20:59:48.371619+0000","last_became_active":"2024-09-10T20:59:47.030792+0000","last_became_peered":"2024-09-10T20:59:47.030792+0000","last_unstale":"2024-09-10T20:59:48.371619+0000","last_undegraded":"2024-09-10T20:59:48.371619+0000","last_fullsized":"2024-09-10T20:59:48.371619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:56.934193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.997504+0000","last_change":"2024-09-10T20:59:44.997504+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.997504+0000","last_undegraded":"2024-09-10T20:59:44.997504+0000","last_fullsized":"2024-09-10T20:59:44.997504+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.995527+0000","last_change":"2024-09-10T20:59:44.995527+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.995527+0000","last_undegraded":"2024-09-10T20:59:44.995527+0000","last_fullsized":"2024-09-10T20:59:44.995527+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008062+0000","last_change":"2024-09-10T20:59:46.012810+0000","last_active":"2024-09-10T20:59:48.008062+0000","last_peered":"2024-09-10T20:59:48.008062+0000","last_clean":"2024-09-10T20:59:48.008062+0000","last_became_active":"2024-09-10T20:59:46.012532+0000","last_became_peered":"2024-09-10T20:59:46.012532+0000","last_unstale":"2024-09-10T20:59:48.008062+0000","last_undegraded":"2024-09-10T20:59:48.008062+0000","last_fullsized":"2024-09-10T20:59:48.008062+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:25.702550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009911+0000","last_change":"2024-09-10T20:59:47.032222+0000","last_active":"2024-09-10T20:59:48.009911+0000","last_peered":"2024-09-10T20:59:48.009911+0000","last_clean":"2024-09-10T20:59:48.009911+0000","last_became_active":"2024-09-10T20:59:47.031463+0000","last_became_peered":"2024-09-10T20:59:47.031463+0000","last_unstale":"2024-09-10T20:59:48.009911+0000","last_undegraded":"2024-09-10T20:59:48.009911+0000","last_fullsized":"2024-09-10T20:59:48.009911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:24.386660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.998797+0000","last_change":"2024-09-10T20:59:44.998797+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.998797+0000","last_undegraded":"2024-09-10T20:59:44.998797+0000","last_fullsized":"2024-09-10T20:59:44.998797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":140,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T20:59:44.999128+0000","last_change":"2024-09-10T20:58:36.895993+0000","last_active":"2024-09-10T20:59:44.999128+0000","last_peered":"2024-09-10T20:59:44.999128+0000","last_clean":"2024-09-10T20:59:44.999128+0000","last_became_active":"2024-09-10T20:58:36.894988+0000","last_became_peered":"2024-09-10T20:58:36.894988+0000","last_unstale":"2024-09-10T20:59:44.999128+0000","last_undegraded":"2024-09-10T20:59:44.999128+0000","last_fullsized":"2024-09-10T20:59:44.999128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_clean_scrub_stamp":"2024-09-10T20:58:35.588702+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.082528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":82,"acting":82,"num_store_stats":5},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":767894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":177526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":177526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":471040,"data_stored":199698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":198122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":198142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1052672,"data_stored":788454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":198164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:59:50.988 INFO:tasks.ceph.ceph_manager.ceph:PG 6.32 is not active+clean 2024-09-10T20:59:50.988 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.988 INFO:tasks.ceph.ceph_manager.ceph:PG 6.30 is not active+clean 2024-09-10T20:59:50.989 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.989 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2f is not active+clean 2024-09-10T20:59:50.989 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.989 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2e is not active+clean 2024-09-10T20:59:50.989 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.989 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2c is not active+clean 2024-09-10T20:59:50.989 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.989 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2a is not active+clean 2024-09-10T20:59:50.989 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.989 INFO:tasks.ceph.ceph_manager.ceph:PG 6.29 is not active+clean 2024-09-10T20:59:50.989 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.990 INFO:tasks.ceph.ceph_manager.ceph:PG 6.27 is not active+clean 2024-09-10T20:59:50.990 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.990 INFO:tasks.ceph.ceph_manager.ceph:PG 6.25 is not active+clean 2024-09-10T20:59:50.990 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.990 INFO:tasks.ceph.ceph_manager.ceph:PG 6.23 is not active+clean 2024-09-10T20:59:50.990 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.990 INFO:tasks.ceph.ceph_manager.ceph:PG 6.22 is not active+clean 2024-09-10T20:59:50.990 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.990 INFO:tasks.ceph.ceph_manager.ceph:PG 6.21 is not active+clean 2024-09-10T20:59:50.990 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.990 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1c is not active+clean 2024-09-10T20:59:50.990 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.991 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1f is not active+clean 2024-09-10T20:59:50.991 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-10T20:59:44.997501+0000', 'last_change': '2024-09-10T20:59:44.997501+0000', 'last_active': '2024-09-10T20:59:44.986452+0000', 'last_peered': '2024-09-10T20:59:44.986452+0000', 'last_clean': '2024-09-10T20:59:44.986452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:44.997501+0000', 'last_undegraded': '2024-09-10T20:59:44.997501+0000', 'last_fullsized': '2024-09-10T20:59:44.997501+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T20:59:50.991 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1e is not active+clean 2024-09-10T20:59:50.991 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.991 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1c is not active+clean 2024-09-10T20:59:50.991 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-10T20:59:44.995186+0000', 'last_change': '2024-09-10T20:59:44.995186+0000', 'last_active': '2024-09-10T20:59:44.986452+0000', 'last_peered': '2024-09-10T20:59:44.986452+0000', 'last_clean': '2024-09-10T20:59:44.986452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:44.995186+0000', 'last_undegraded': '2024-09-10T20:59:44.995186+0000', 'last_fullsized': '2024-09-10T20:59:44.995186+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T20:59:50.991 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1a is not active+clean 2024-09-10T20:59:50.991 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-10T20:59:44.999305+0000', 'last_change': '2024-09-10T20:59:44.999305+0000', 'last_active': '2024-09-10T20:59:44.986452+0000', 'last_peered': '2024-09-10T20:59:44.986452+0000', 'last_clean': '2024-09-10T20:59:44.986452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:44.999305+0000', 'last_undegraded': '2024-09-10T20:59:44.999305+0000', 'last_fullsized': '2024-09-10T20:59:44.999305+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T20:59:50.991 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1a is not active+clean 2024-09-10T20:59:50.991 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.991 INFO:tasks.ceph.ceph_manager.ceph:PG 5.18 is not active+clean 2024-09-10T20:59:50.992 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-10T20:59:44.994818+0000', 'last_change': '2024-09-10T20:59:44.994818+0000', 'last_active': '2024-09-10T20:59:44.986452+0000', 'last_peered': '2024-09-10T20:59:44.986452+0000', 'last_clean': '2024-09-10T20:59:44.986452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:44.994818+0000', 'last_undegraded': '2024-09-10T20:59:44.994818+0000', 'last_fullsized': '2024-09-10T20:59:44.994818+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T20:59:50.992 INFO:tasks.ceph.ceph_manager.ceph:PG 6.15 is not active+clean 2024-09-10T20:59:50.992 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.992 INFO:tasks.ceph.ceph_manager.ceph:PG 6.17 is not active+clean 2024-09-10T20:59:50.992 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.992 INFO:tasks.ceph.ceph_manager.ceph:PG 6.12 is not active+clean 2024-09-10T20:59:50.992 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.992 INFO:tasks.ceph.ceph_manager.ceph:PG 5.11 is not active+clean 2024-09-10T20:59:50.992 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-10T20:59:44.997501+0000', 'last_change': '2024-09-10T20:59:44.997501+0000', 'last_active': '2024-09-10T20:59:44.986452+0000', 'last_peered': '2024-09-10T20:59:44.986452+0000', 'last_clean': '2024-09-10T20:59:44.986452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:44.997501+0000', 'last_undegraded': '2024-09-10T20:59:44.997501+0000', 'last_fullsized': '2024-09-10T20:59:44.997501+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T20:59:50.992 INFO:tasks.ceph.ceph_manager.ceph:PG 5.10 is not active+clean 2024-09-10T20:59:50.992 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-10T20:59:44.998837+0000', 'last_change': '2024-09-10T20:59:44.998837+0000', 'last_active': '2024-09-10T20:59:44.986452+0000', 'last_peered': '2024-09-10T20:59:44.986452+0000', 'last_clean': '2024-09-10T20:59:44.986452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:44.998837+0000', 'last_undegraded': '2024-09-10T20:59:44.998837+0000', 'last_fullsized': '2024-09-10T20:59:44.998837+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T20:59:50.993 INFO:tasks.ceph.ceph_manager.ceph:PG 5.e is not active+clean 2024-09-10T20:59:50.993 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-10T20:59:44.995492+0000', 'last_change': '2024-09-10T20:59:44.995492+0000', 'last_active': '2024-09-10T20:59:44.986452+0000', 'last_peered': '2024-09-10T20:59:44.986452+0000', 'last_clean': '2024-09-10T20:59:44.986452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:44.995492+0000', 'last_undegraded': '2024-09-10T20:59:44.995492+0000', 'last_fullsized': '2024-09-10T20:59:44.995492+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T20:59:50.993 INFO:tasks.ceph.ceph_manager.ceph:PG 6.e is not active+clean 2024-09-10T20:59:50.993 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.993 INFO:tasks.ceph.ceph_manager.ceph:PG 6.8 is not active+clean 2024-09-10T20:59:50.993 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.993 INFO:tasks.ceph.ceph_manager.ceph:PG 5.9 is not active+clean 2024-09-10T20:59:50.993 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-10T20:59:44.998868+0000', 'last_change': '2024-09-10T20:59:44.998868+0000', 'last_active': '2024-09-10T20:59:44.986452+0000', 'last_peered': '2024-09-10T20:59:44.986452+0000', 'last_clean': '2024-09-10T20:59:44.986452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:44.998868+0000', 'last_undegraded': '2024-09-10T20:59:44.998868+0000', 'last_fullsized': '2024-09-10T20:59:44.998868+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T20:59:50.993 INFO:tasks.ceph.ceph_manager.ceph:PG 6.5 is not active+clean 2024-09-10T20:59:50.993 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.993 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2 is not active+clean 2024-09-10T20:59:50.994 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-10T20:59:44.997504+0000', 'last_change': '2024-09-10T20:59:44.997504+0000', 'last_active': '2024-09-10T20:59:44.986452+0000', 'last_peered': '2024-09-10T20:59:44.986452+0000', 'last_clean': '2024-09-10T20:59:44.986452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:44.997504+0000', 'last_undegraded': '2024-09-10T20:59:44.997504+0000', 'last_fullsized': '2024-09-10T20:59:44.997504+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T20:59:50.994 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2 is not active+clean 2024-09-10T20:59:50.994 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.994 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1 is not active+clean 2024-09-10T20:59:50.994 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-10T20:59:44.995527+0000', 'last_change': '2024-09-10T20:59:44.995527+0000', 'last_active': '2024-09-10T20:59:44.986452+0000', 'last_peered': '2024-09-10T20:59:44.986452+0000', 'last_clean': '2024-09-10T20:59:44.986452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:44.995527+0000', 'last_undegraded': '2024-09-10T20:59:44.995527+0000', 'last_fullsized': '2024-09-10T20:59:44.995527+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T20:59:50.994 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3 is not active+clean 2024-09-10T20:59:50.994 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T20:59:45.991434+0000', 'last_change': '2024-09-10T20:59:45.991434+0000', 'last_active': '2024-09-10T20:59:45.991434+0000', 'last_peered': '2024-09-10T20:59:45.991434+0000', 'last_clean': '2024-09-10T20:59:45.991434+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:45.991434+0000', 'last_undegraded': '2024-09-10T20:59:45.991434+0000', 'last_fullsized': '2024-09-10T20:59:45.991434+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:45.991434+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T20:59:50.994 INFO:tasks.ceph.ceph_manager.ceph:PG 5.4 is not active+clean 2024-09-10T20:59:50.994 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-10T20:59:44.998797+0000', 'last_change': '2024-09-10T20:59:44.998797+0000', 'last_active': '2024-09-10T20:59:44.986452+0000', 'last_peered': '2024-09-10T20:59:44.986452+0000', 'last_clean': '2024-09-10T20:59:44.986452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T20:59:44.998797+0000', 'last_undegraded': '2024-09-10T20:59:44.998797+0000', 'last_fullsized': '2024-09-10T20:59:44.998797+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'last_clean_scrub_stamp': '2024-09-10T20:59:44.986452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T20:59:50.995 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T20:59:51.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.121+0000 7f5cdec85640 1 -- 172.21.15.98:0/341257130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8074c70 msgr2=0x7f5cd80cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:51.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.121+0000 7f5cdec85640 1 --2- 172.21.15.98:0/341257130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8074c70 0x7f5cd80cdef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005fd0 tx=0x7f5cd002a8b0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.121+0000 7f5cdec85640 1 -- 172.21.15.98:0/341257130 shutdown_connections 2024-09-10T20:59:51.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.121+0000 7f5cdec85640 1 --2- 172.21.15.98:0/341257130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8074c70 0x7f5cd80cdef0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.121+0000 7f5cdec85640 1 --2- 172.21.15.98:0/341257130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cd806d160 0x7f5cd80b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.121+0000 7f5cdec85640 1 -- 172.21.15.98:0/341257130 >> 172.21.15.98:0/341257130 conn(0x7f5cd8074150 msgr2=0x7f5cd8074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:51.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.121+0000 7f5cdec85640 1 -- 172.21.15.98:0/341257130 shutdown_connections 2024-09-10T20:59:51.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.121+0000 7f5cdec85640 1 -- 172.21.15.98:0/341257130 wait complete. 2024-09-10T20:59:51.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.122+0000 7f5cdec85640 1 Processor -- start 2024-09-10T20:59:51.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.122+0000 7f5cdec85640 1 -- start start 2024-09-10T20:59:51.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.123+0000 7f5cdec85640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cd806d160 0x7f5cd80c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:51.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.123+0000 7f5cdec85640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8074c70 0x7f5cd80ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:51.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.123+0000 7f5cdec85640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cd815e0f0 0x7f5cd80ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:51.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.123+0000 7f5cdec85640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd80697b0 con 0x7f5cd815e0f0 2024-09-10T20:59:51.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.123+0000 7f5cdec85640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd8069630 con 0x7f5cd8074c70 2024-09-10T20:59:51.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.123+0000 7f5cdec85640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5cd80ce430 con 0x7f5cd806d160 2024-09-10T20:59:51.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.123+0000 7f5cdd222640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cd815e0f0 0x7f5cd80ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:51.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.123+0000 7f5cdd222640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cd815e0f0 0x7f5cd80ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54834/0 (socket says 172.21.15.98:54834) 2024-09-10T20:59:51.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.123+0000 7f5cdd222640 1 -- 172.21.15.98:0/282115813 learned_addr learned my addr 172.21.15.98:0/282115813 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:51.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.124+0000 7f5cdca21640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cd806d160 0x7f5cd80c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:51.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.124+0000 7f5cd7fff640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8074c70 0x7f5cd80ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:51.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.124+0000 7f5cdd222640 1 -- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cd806d160 msgr2=0x7f5cd80c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:51.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.124+0000 7f5cdd222640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cd806d160 0x7f5cd80c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.124+0000 7f5cdd222640 1 -- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8074c70 msgr2=0x7f5cd80ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:51.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.124+0000 7f5cdd222640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8074c70 0x7f5cd80ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.124+0000 7f5cdd222640 1 -- 172.21.15.98:0/282115813 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd80cb300 con 0x7f5cd815e0f0 2024-09-10T20:59:51.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.124+0000 7f5cd7fff640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8074c70 0x7f5cd80ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:51.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.124+0000 7f5cdd222640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cd815e0f0 0x7f5cd80ad400 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0006010 tx=0x7f5cd00316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:51.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.124+0000 7f5cd57fa640 1 -- 172.21.15.98:0/282115813 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0040070 con 0x7f5cd815e0f0 2024-09-10T20:59:51.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.124+0000 7f5cd57fa640 1 -- 172.21.15.98:0/282115813 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cd0004040 con 0x7f5cd815e0f0 2024-09-10T20:59:51.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.124+0000 7f5cd57fa640 1 -- 172.21.15.98:0/282115813 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0004450 con 0x7f5cd815e0f0 2024-09-10T20:59:51.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.124+0000 7f5cdec85640 1 -- 172.21.15.98:0/282115813 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd80cb5e0 con 0x7f5cd815e0f0 2024-09-10T20:59:51.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.124+0000 7f5cdec85640 1 -- 172.21.15.98:0/282115813 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd80cae90 con 0x7f5cd815e0f0 2024-09-10T20:59:51.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.125+0000 7f5cdec85640 1 -- 172.21.15.98:0/282115813 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd80ce930 con 0x7f5cd815e0f0 2024-09-10T20:59:51.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.126+0000 7f5cd57fa640 1 -- 172.21.15.98:0/282115813 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5cd00045f0 con 0x7f5cd815e0f0 2024-09-10T20:59:51.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.126+0000 7f5cd57fa640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5cbc077f70 0x7f5cbc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:51.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.126+0000 7f5cd57fa640 1 -- 172.21.15.98:0/282115813 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f5cd00bb6c0 con 0x7f5cd815e0f0 2024-09-10T20:59:51.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.126+0000 7f5cdca21640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5cbc077f70 0x7f5cbc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:51.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.127+0000 7f5cdca21640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5cbc077f70 0x7f5cbc07a430 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc001720 tx=0x7f5ccc007e90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:51.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.130+0000 7f5cd57fa640 1 -- 172.21.15.98:0/282115813 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5cd0084040 con 0x7f5cd815e0f0 2024-09-10T20:59:51.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.373+0000 7f5cdec85640 1 -- 172.21.15.98:0/282115813 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5cd815d7c0 con 0x7f5cd815e0f0 2024-09-10T20:59:51.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.374+0000 7f5cd57fa640 1 -- 172.21.15.98:0/282115813 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1764 (secure 0 0 0) 0x7f5cd0083940 con 0x7f5cd815e0f0 2024-09-10T20:59:51.378 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:51.378 INFO:teuthology.orchestra.run.smithi098.stdout:{"fsid":"41041377-a016-49fd-be1f-323d0b834557","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":88,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":40,"num_osds":8,"num_up_osds":8,"osd_up_since":1726001915,"num_in_osds":8,"osd_in_since":1726001905,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":63},{"state_name":"unknown","count":23},{"state_name":"creating+peering","count":11}],"num_pgs":97,"num_pools":3,"num_objects":16,"data_bytes":592196,"bytes_used":226172928,"bytes_avail":772867940352,"bytes_total":773094113280,"unknown_pgs_ratio":0.23711340129375458,"inactive_pgs_ratio":0.11340206116437912,"write_bytes_sec":2245,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":17,"btime":"2024-09-10T20:59:49:345393+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5001}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-10T20:59:36.918686+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T20:59:51.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.376+0000 7f5cdec85640 1 -- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5cbc077f70 msgr2=0x7f5cbc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:51.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.376+0000 7f5cdec85640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5cbc077f70 0x7f5cbc07a430 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc001720 tx=0x7f5ccc007e90 comp rx=0 tx=0).stop 2024-09-10T20:59:51.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.376+0000 7f5cdec85640 1 -- 172.21.15.98:0/282115813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cd815e0f0 msgr2=0x7f5cd80ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:51.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.376+0000 7f5cdec85640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cd815e0f0 0x7f5cd80ad400 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0006010 tx=0x7f5cd00316e0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.376+0000 7f5cdec85640 1 -- 172.21.15.98:0/282115813 shutdown_connections 2024-09-10T20:59:51.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.376+0000 7f5cdec85640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5cbc077f70 0x7f5cbc07a430 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.376+0000 7f5cdec85640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5cd815e0f0 0x7f5cd80ad400 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.376+0000 7f5cdec85640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5cd8074c70 0x7f5cd80ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.376+0000 7f5cdec85640 1 --2- 172.21.15.98:0/282115813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5cd806d160 0x7f5cd80c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.376+0000 7f5cdec85640 1 -- 172.21.15.98:0/282115813 >> 172.21.15.98:0/282115813 conn(0x7f5cd8074150 msgr2=0x7f5cd8074530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:51.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.376+0000 7f5cdec85640 1 -- 172.21.15.98:0/282115813 shutdown_connections 2024-09-10T20:59:51.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.376+0000 7f5cdec85640 1 -- 172.21.15.98:0/282115813 wait complete. 2024-09-10T20:59:51.391 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T20:59:51.391 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:59:51.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.487+0000 7f1bdc59b640 1 -- 172.21.15.98:0/3726549397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bd406cb70 msgr2=0x7f1bd40caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:51.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.487+0000 7f1bdc59b640 1 --2- 172.21.15.98:0/3726549397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bd406cb70 0x7f1bd40caeb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1bd406d450 tx=0x7f1bcc02a8f0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.487+0000 7f1bdc59b640 1 -- 172.21.15.98:0/3726549397 shutdown_connections 2024-09-10T20:59:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.487+0000 7f1bdc59b640 1 --2- 172.21.15.98:0/3726549397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bd406cb70 0x7f1bd40caeb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.487+0000 7f1bdc59b640 1 --2- 172.21.15.98:0/3726549397 >> v2:172.21.15.190:3300/0 conn(0x7f1bd406e0a0 0x7f1bd40ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.487+0000 7f1bdc59b640 1 --2- 172.21.15.98:0/3726549397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bd406dcc0 0x7f1bd40b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.487+0000 7f1bdc59b640 1 -- 172.21.15.98:0/3726549397 >> 172.21.15.98:0/3726549397 conn(0x7f1bd4074150 msgr2=0x7f1bd4074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.487+0000 7f1bdc59b640 1 -- 172.21.15.98:0/3726549397 shutdown_connections 2024-09-10T20:59:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.487+0000 7f1bdc59b640 1 -- 172.21.15.98:0/3726549397 wait complete. 2024-09-10T20:59:51.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.488+0000 7f1bdc59b640 1 Processor -- start 2024-09-10T20:59:51.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.488+0000 7f1bdc59b640 1 -- start start 2024-09-10T20:59:51.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bdc59b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bd406cb70 0x7f1bd415b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:51.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bdc59b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bd406dcc0 0x7f1bd40c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:51.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bdc59b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bd406e0a0 0x7f1bd40c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:51.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bdc59b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd40cb570 con 0x7f1bd406cb70 2024-09-10T20:59:51.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bdc59b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd40cb3f0 con 0x7f1bd406e0a0 2024-09-10T20:59:51.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bdc59b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd415eef0 con 0x7f1bd406dcc0 2024-09-10T20:59:51.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bda337640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bd406cb70 0x7f1bd415b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:51.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bda337640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bd406cb70 0x7f1bd415b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54862/0 (socket says 172.21.15.98:54862) 2024-09-10T20:59:51.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bda337640 1 -- 172.21.15.98:0/1088032328 learned_addr learned my addr 172.21.15.98:0/1088032328 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:51.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bd9b36640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bd406dcc0 0x7f1bd40c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:51.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bdab38640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bd406e0a0 0x7f1bd40c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:51.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bda337640 1 -- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bd406dcc0 msgr2=0x7f1bd40c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:51.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bda337640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bd406dcc0 0x7f1bd40c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bda337640 1 -- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bd406e0a0 msgr2=0x7f1bd40c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:51.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bda337640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bd406e0a0 0x7f1bd40c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bda337640 1 -- 172.21.15.98:0/1088032328 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd415bb60 con 0x7f1bd406cb70 2024-09-10T20:59:51.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bdab38640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bd406e0a0 0x7f1bd40c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:51.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bda337640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bd406cb70 0x7f1bd415b160 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000c920 tx=0x7f1bd000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:51.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bcaffd640 1 -- 172.21.15.98:0/1088032328 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0007c70 con 0x7f1bd406cb70 2024-09-10T20:59:51.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bcaffd640 1 -- 172.21.15.98:0/1088032328 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bd000f450 con 0x7f1bd406cb70 2024-09-10T20:59:51.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bdc59b640 1 -- 172.21.15.98:0/1088032328 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd415be40 con 0x7f1bd406cb70 2024-09-10T20:59:51.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bcaffd640 1 -- 172.21.15.98:0/1088032328 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd000f8a0 con 0x7f1bd406cb70 2024-09-10T20:59:51.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.489+0000 7f1bdc59b640 1 -- 172.21.15.98:0/1088032328 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd415c400 con 0x7f1bd406cb70 2024-09-10T20:59:51.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.490+0000 7f1bdc59b640 1 -- 172.21.15.98:0/1088032328 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd40b1740 con 0x7f1bd406cb70 2024-09-10T20:59:51.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.491+0000 7f1bcaffd640 1 -- 172.21.15.98:0/1088032328 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1bd0020030 con 0x7f1bd406cb70 2024-09-10T20:59:51.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.491+0000 7f1bcaffd640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1bb8077fa0 0x7f1bb807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:51.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.492+0000 7f1bcaffd640 1 -- 172.21.15.98:0/1088032328 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f1bd009a470 con 0x7f1bd406cb70 2024-09-10T20:59:51.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.492+0000 7f1bd9b36640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1bb8077fa0 0x7f1bb807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:51.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.492+0000 7f1bd9b36640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1bb8077fa0 0x7f1bb807a460 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4005fd0 tx=0x7f1bc400d490 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:51.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.496+0000 7f1bcaffd640 1 -- 172.21.15.98:0/1088032328 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1bd0014030 con 0x7f1bd406cb70 2024-09-10T20:59:51.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.667+0000 7f1bdc59b640 1 -- 172.21.15.98:0/1088032328 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1bd40b6010 con 0x7f1bb8077fa0 2024-09-10T20:59:51.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.672+0000 7f1bcaffd640 1 -- 172.21.15.98:0/1088032328 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247672 (secure 0 0 0) 0x7f1bd40b6010 con 0x7f1bb8077fa0 2024-09-10T20:59:51.678 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:51.680 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-10T20:59:51.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.675+0000 7f1bdc59b640 1 -- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1bb8077fa0 msgr2=0x7f1bb807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:51.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.675+0000 7f1bdc59b640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1bb8077fa0 0x7f1bb807a460 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4005fd0 tx=0x7f1bc400d490 comp rx=0 tx=0).stop 2024-09-10T20:59:51.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.675+0000 7f1bdc59b640 1 -- 172.21.15.98:0/1088032328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bd406cb70 msgr2=0x7f1bd415b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:51.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.675+0000 7f1bdc59b640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bd406cb70 0x7f1bd415b160 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000c920 tx=0x7f1bd000cde0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.675+0000 7f1bdc59b640 1 -- 172.21.15.98:0/1088032328 shutdown_connections 2024-09-10T20:59:51.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.675+0000 7f1bdc59b640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1bb8077fa0 0x7f1bb807a460 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.675+0000 7f1bdc59b640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bd406e0a0 0x7f1bd40c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.675+0000 7f1bdc59b640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bd406dcc0 0x7f1bd40c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.675+0000 7f1bdc59b640 1 --2- 172.21.15.98:0/1088032328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bd406cb70 0x7f1bd415b160 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:51.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.675+0000 7f1bdc59b640 1 -- 172.21.15.98:0/1088032328 >> 172.21.15.98:0/1088032328 conn(0x7f1bd4074150 msgr2=0x7f1bd41656b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:51.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.675+0000 7f1bdc59b640 1 -- 172.21.15.98:0/1088032328 shutdown_connections 2024-09-10T20:59:51.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:51.675+0000 7f1bdc59b640 1 -- 172.21.15.98:0/1088032328 wait complete. 2024-09-10T20:59:51.690 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-09-10T20:59:50.921570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":124,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94,"ondisk_log_size":94,"up":148,"acting":148,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220872,"kb_used_data":4640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753848,"statfs":{"total":773094113280,"available":772867940352,"internally_reserved":0,"allocated":4751360,"data_stored":2705526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.928858"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008495+0000","last_change":"2024-09-10T20:59:47.020403+0000","last_active":"2024-09-10T20:59:48.008495+0000","last_peered":"2024-09-10T20:59:48.008495+0000","last_clean":"2024-09-10T20:59:48.008495+0000","last_became_active":"2024-09-10T20:59:47.018947+0000","last_became_peered":"2024-09-10T20:59:47.018947+0000","last_unstale":"2024-09-10T20:59:48.008495+0000","last_undegraded":"2024-09-10T20:59:48.008495+0000","last_fullsized":"2024-09-10T20:59:48.008495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:41.452362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008383+0000","last_change":"2024-09-10T20:59:47.022856+0000","last_active":"2024-09-10T20:59:48.008383+0000","last_peered":"2024-09-10T20:59:48.008383+0000","last_clean":"2024-09-10T20:59:48.008383+0000","last_became_active":"2024-09-10T20:59:47.022579+0000","last_became_peered":"2024-09-10T20:59:47.022579+0000","last_unstale":"2024-09-10T20:59:48.008383+0000","last_undegraded":"2024-09-10T20:59:48.008383+0000","last_fullsized":"2024-09-10T20:59:48.008383+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:51.434814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371766+0000","last_change":"2024-09-10T20:59:47.028091+0000","last_active":"2024-09-10T20:59:48.371766+0000","last_peered":"2024-09-10T20:59:48.371766+0000","last_clean":"2024-09-10T20:59:48.371766+0000","last_became_active":"2024-09-10T20:59:47.027830+0000","last_became_peered":"2024-09-10T20:59:47.027830+0000","last_unstale":"2024-09-10T20:59:48.371766+0000","last_undegraded":"2024-09-10T20:59:48.371766+0000","last_fullsized":"2024-09-10T20:59:48.371766+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:17.875896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024599+0000","last_change":"2024-09-10T20:59:47.033657+0000","last_active":"2024-09-10T20:59:48.024599+0000","last_peered":"2024-09-10T20:59:48.024599+0000","last_clean":"2024-09-10T20:59:48.024599+0000","last_became_active":"2024-09-10T20:59:47.033363+0000","last_became_peered":"2024-09-10T20:59:47.033363+0000","last_unstale":"2024-09-10T20:59:48.024599+0000","last_undegraded":"2024-09-10T20:59:48.024599+0000","last_fullsized":"2024-09-10T20:59:48.024599+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:45.439857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024571+0000","last_change":"2024-09-10T20:59:47.030433+0000","last_active":"2024-09-10T20:59:48.024571+0000","last_peered":"2024-09-10T20:59:48.024571+0000","last_clean":"2024-09-10T20:59:48.024571+0000","last_became_active":"2024-09-10T20:59:47.030093+0000","last_became_peered":"2024-09-10T20:59:47.030093+0000","last_unstale":"2024-09-10T20:59:48.024571+0000","last_undegraded":"2024-09-10T20:59:48.024571+0000","last_fullsized":"2024-09-10T20:59:48.024571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:23.601594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371305+0000","last_change":"2024-09-10T20:59:47.022075+0000","last_active":"2024-09-10T20:59:48.371305+0000","last_peered":"2024-09-10T20:59:48.371305+0000","last_clean":"2024-09-10T20:59:48.371305+0000","last_became_active":"2024-09-10T20:59:47.021810+0000","last_became_peered":"2024-09-10T20:59:47.021810+0000","last_unstale":"2024-09-10T20:59:48.371305+0000","last_undegraded":"2024-09-10T20:59:48.371305+0000","last_fullsized":"2024-09-10T20:59:48.371305+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:53.763319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009198+0000","last_change":"2024-09-10T20:59:47.024763+0000","last_active":"2024-09-10T20:59:48.009198+0000","last_peered":"2024-09-10T20:59:48.009198+0000","last_clean":"2024-09-10T20:59:48.009198+0000","last_became_active":"2024-09-10T20:59:47.024475+0000","last_became_peered":"2024-09-10T20:59:47.024475+0000","last_unstale":"2024-09-10T20:59:48.009198+0000","last_undegraded":"2024-09-10T20:59:48.009198+0000","last_fullsized":"2024-09-10T20:59:48.009198+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:51:57.920889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008071+0000","last_change":"2024-09-10T20:59:47.026033+0000","last_active":"2024-09-10T20:59:48.008071+0000","last_peered":"2024-09-10T20:59:48.008071+0000","last_clean":"2024-09-10T20:59:48.008071+0000","last_became_active":"2024-09-10T20:59:47.025434+0000","last_became_peered":"2024-09-10T20:59:47.025434+0000","last_unstale":"2024-09-10T20:59:48.008071+0000","last_undegraded":"2024-09-10T20:59:48.008071+0000","last_fullsized":"2024-09-10T20:59:48.008071+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:28.753205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371967+0000","last_change":"2024-09-10T20:59:47.028797+0000","last_active":"2024-09-10T20:59:48.371967+0000","last_peered":"2024-09-10T20:59:48.371967+0000","last_clean":"2024-09-10T20:59:48.371967+0000","last_became_active":"2024-09-10T20:59:47.028522+0000","last_became_peered":"2024-09-10T20:59:47.028522+0000","last_unstale":"2024-09-10T20:59:48.371967+0000","last_undegraded":"2024-09-10T20:59:48.371967+0000","last_fullsized":"2024-09-10T20:59:48.371967+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:35.047921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010611+0000","last_change":"2024-09-10T20:59:47.028365+0000","last_active":"2024-09-10T20:59:48.010611+0000","last_peered":"2024-09-10T20:59:48.010611+0000","last_clean":"2024-09-10T20:59:48.010611+0000","last_became_active":"2024-09-10T20:59:47.028064+0000","last_became_peered":"2024-09-10T20:59:47.028064+0000","last_unstale":"2024-09-10T20:59:48.010611+0000","last_undegraded":"2024-09-10T20:59:48.010611+0000","last_fullsized":"2024-09-10T20:59:48.010611+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:39.938414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009375+0000","last_change":"2024-09-10T20:59:47.031189+0000","last_active":"2024-09-10T20:59:48.009375+0000","last_peered":"2024-09-10T20:59:48.009375+0000","last_clean":"2024-09-10T20:59:48.009375+0000","last_became_active":"2024-09-10T20:59:47.030852+0000","last_became_peered":"2024-09-10T20:59:47.030852+0000","last_unstale":"2024-09-10T20:59:48.009375+0000","last_undegraded":"2024-09-10T20:59:48.009375+0000","last_fullsized":"2024-09-10T20:59:48.009375+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:49.659631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371334+0000","last_change":"2024-09-10T20:59:47.024031+0000","last_active":"2024-09-10T20:59:48.371334+0000","last_peered":"2024-09-10T20:59:48.371334+0000","last_clean":"2024-09-10T20:59:48.371334+0000","last_became_active":"2024-09-10T20:59:47.023695+0000","last_became_peered":"2024-09-10T20:59:47.023695+0000","last_unstale":"2024-09-10T20:59:48.371334+0000","last_undegraded":"2024-09-10T20:59:48.371334+0000","last_fullsized":"2024-09-10T20:59:48.371334+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:09.439607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009786+0000","last_change":"2024-09-10T20:59:47.028762+0000","last_active":"2024-09-10T20:59:48.009786+0000","last_peered":"2024-09-10T20:59:48.009786+0000","last_clean":"2024-09-10T20:59:48.009786+0000","last_became_active":"2024-09-10T20:59:47.028406+0000","last_became_peered":"2024-09-10T20:59:47.028406+0000","last_unstale":"2024-09-10T20:59:48.009786+0000","last_undegraded":"2024-09-10T20:59:48.009786+0000","last_fullsized":"2024-09-10T20:59:48.009786+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:12.795591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009388+0000","last_change":"2024-09-10T20:59:47.024046+0000","last_active":"2024-09-10T20:59:48.009388+0000","last_peered":"2024-09-10T20:59:48.009388+0000","last_clean":"2024-09-10T20:59:48.009388+0000","last_became_active":"2024-09-10T20:59:47.023745+0000","last_became_peered":"2024-09-10T20:59:47.023745+0000","last_unstale":"2024-09-10T20:59:48.009388+0000","last_undegraded":"2024-09-10T20:59:48.009388+0000","last_fullsized":"2024-09-10T20:59:48.009388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:32.861565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024695+0000","last_change":"2024-09-10T20:59:47.030399+0000","last_active":"2024-09-10T20:59:48.024695+0000","last_peered":"2024-09-10T20:59:48.024695+0000","last_clean":"2024-09-10T20:59:48.024695+0000","last_became_active":"2024-09-10T20:59:47.029699+0000","last_became_peered":"2024-09-10T20:59:47.029699+0000","last_unstale":"2024-09-10T20:59:48.024695+0000","last_undegraded":"2024-09-10T20:59:48.024695+0000","last_fullsized":"2024-09-10T20:59:48.024695+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:26.745564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009982+0000","last_change":"2024-09-10T20:59:47.028032+0000","last_active":"2024-09-10T20:59:48.009982+0000","last_peered":"2024-09-10T20:59:48.009982+0000","last_clean":"2024-09-10T20:59:48.009982+0000","last_became_active":"2024-09-10T20:59:47.027708+0000","last_became_peered":"2024-09-10T20:59:47.027708+0000","last_unstale":"2024-09-10T20:59:48.009982+0000","last_undegraded":"2024-09-10T20:59:48.009982+0000","last_fullsized":"2024-09-10T20:59:48.009982+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:48.185509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024474+0000","last_change":"2024-09-10T20:59:47.027243+0000","last_active":"2024-09-10T20:59:48.024474+0000","last_peered":"2024-09-10T20:59:48.024474+0000","last_clean":"2024-09-10T20:59:48.024474+0000","last_became_active":"2024-09-10T20:59:47.026976+0000","last_became_peered":"2024-09-10T20:59:47.026976+0000","last_unstale":"2024-09-10T20:59:48.024474+0000","last_undegraded":"2024-09-10T20:59:48.024474+0000","last_fullsized":"2024-09-10T20:59:48.024474+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:17.556094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371446+0000","last_change":"2024-09-10T20:59:47.029424+0000","last_active":"2024-09-10T20:59:48.371446+0000","last_peered":"2024-09-10T20:59:48.371446+0000","last_clean":"2024-09-10T20:59:48.371446+0000","last_became_active":"2024-09-10T20:59:47.029180+0000","last_became_peered":"2024-09-10T20:59:47.029180+0000","last_unstale":"2024-09-10T20:59:48.371446+0000","last_undegraded":"2024-09-10T20:59:48.371446+0000","last_fullsized":"2024-09-10T20:59:48.371446+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:31.373458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010216+0000","last_change":"2024-09-10T20:59:47.025305+0000","last_active":"2024-09-10T20:59:48.010216+0000","last_peered":"2024-09-10T20:59:48.010216+0000","last_clean":"2024-09-10T20:59:48.010216+0000","last_became_active":"2024-09-10T20:59:47.025036+0000","last_became_peered":"2024-09-10T20:59:47.025036+0000","last_unstale":"2024-09-10T20:59:48.010216+0000","last_undegraded":"2024-09-10T20:59:48.010216+0000","last_fullsized":"2024-09-10T20:59:48.010216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:25.337419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371305+0000","last_change":"2024-09-10T20:59:47.021237+0000","last_active":"2024-09-10T20:59:48.371305+0000","last_peered":"2024-09-10T20:59:48.371305+0000","last_clean":"2024-09-10T20:59:48.371305+0000","last_became_active":"2024-09-10T20:59:47.020964+0000","last_became_peered":"2024-09-10T20:59:47.020964+0000","last_unstale":"2024-09-10T20:59:48.371305+0000","last_undegraded":"2024-09-10T20:59:48.371305+0000","last_fullsized":"2024-09-10T20:59:48.371305+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:50.897033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.997501+0000","last_change":"2024-09-10T20:59:44.997501+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.997501+0000","last_undegraded":"2024-09-10T20:59:44.997501+0000","last_fullsized":"2024-09-10T20:59:44.997501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371836+0000","last_change":"2024-09-10T20:59:47.028678+0000","last_active":"2024-09-10T20:59:48.371836+0000","last_peered":"2024-09-10T20:59:48.371836+0000","last_clean":"2024-09-10T20:59:48.371836+0000","last_became_active":"2024-09-10T20:59:47.028134+0000","last_became_peered":"2024-09-10T20:59:47.028134+0000","last_unstale":"2024-09-10T20:59:48.371836+0000","last_undegraded":"2024-09-10T20:59:48.371836+0000","last_fullsized":"2024-09-10T20:59:48.371836+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:21.297545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371469+0000","last_change":"2024-09-10T20:59:46.014029+0000","last_active":"2024-09-10T20:59:48.371469+0000","last_peered":"2024-09-10T20:59:48.371469+0000","last_clean":"2024-09-10T20:59:48.371469+0000","last_became_active":"2024-09-10T20:59:46.013771+0000","last_became_peered":"2024-09-10T20:59:46.013771+0000","last_unstale":"2024-09-10T20:59:48.371469+0000","last_undegraded":"2024-09-10T20:59:48.371469+0000","last_fullsized":"2024-09-10T20:59:48.371469+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:05.365319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371865+0000","last_change":"2024-09-10T20:59:46.012293+0000","last_active":"2024-09-10T20:59:48.371865+0000","last_peered":"2024-09-10T20:59:48.371865+0000","last_clean":"2024-09-10T20:59:48.371865+0000","last_became_active":"2024-09-10T20:59:46.012014+0000","last_became_peered":"2024-09-10T20:59:46.012014+0000","last_unstale":"2024-09-10T20:59:48.371865+0000","last_undegraded":"2024-09-10T20:59:48.371865+0000","last_fullsized":"2024-09-10T20:59:48.371865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:30.723075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008466+0000","last_change":"2024-09-10T20:59:47.032715+0000","last_active":"2024-09-10T20:59:48.008466+0000","last_peered":"2024-09-10T20:59:48.008466+0000","last_clean":"2024-09-10T20:59:48.008466+0000","last_became_active":"2024-09-10T20:59:47.032448+0000","last_became_peered":"2024-09-10T20:59:47.032448+0000","last_unstale":"2024-09-10T20:59:48.008466+0000","last_undegraded":"2024-09-10T20:59:48.008466+0000","last_fullsized":"2024-09-10T20:59:48.008466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:34.824871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.995186+0000","last_change":"2024-09-10T20:59:44.995186+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.995186+0000","last_undegraded":"2024-09-10T20:59:44.995186+0000","last_fullsized":"2024-09-10T20:59:44.995186+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371308+0000","last_change":"2024-09-10T20:59:47.026327+0000","last_active":"2024-09-10T20:59:48.371308+0000","last_peered":"2024-09-10T20:59:48.371308+0000","last_clean":"2024-09-10T20:59:48.371308+0000","last_became_active":"2024-09-10T20:59:47.026074+0000","last_became_peered":"2024-09-10T20:59:47.026074+0000","last_unstale":"2024-09-10T20:59:48.371308+0000","last_undegraded":"2024-09-10T20:59:48.371308+0000","last_fullsized":"2024-09-10T20:59:48.371308+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:41.166136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010096+0000","last_change":"2024-09-10T20:59:46.014169+0000","last_active":"2024-09-10T20:59:48.010096+0000","last_peered":"2024-09-10T20:59:48.010096+0000","last_clean":"2024-09-10T20:59:48.010096+0000","last_became_active":"2024-09-10T20:59:46.013940+0000","last_became_peered":"2024-09-10T20:59:46.013940+0000","last_unstale":"2024-09-10T20:59:48.010096+0000","last_undegraded":"2024-09-10T20:59:48.010096+0000","last_fullsized":"2024-09-10T20:59:48.010096+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:52.758336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009576+0000","last_change":"2024-09-10T20:59:47.027620+0000","last_active":"2024-09-10T20:59:48.009576+0000","last_peered":"2024-09-10T20:59:48.009576+0000","last_clean":"2024-09-10T20:59:48.009576+0000","last_became_active":"2024-09-10T20:59:47.027339+0000","last_became_peered":"2024-09-10T20:59:47.027339+0000","last_unstale":"2024-09-10T20:59:48.009576+0000","last_undegraded":"2024-09-10T20:59:48.009576+0000","last_fullsized":"2024-09-10T20:59:48.009576+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:37.031813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.999305+0000","last_change":"2024-09-10T20:59:44.999305+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.999305+0000","last_undegraded":"2024-09-10T20:59:44.999305+0000","last_fullsized":"2024-09-10T20:59:44.999305+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371718+0000","last_change":"2024-09-10T20:59:46.012513+0000","last_active":"2024-09-10T20:59:48.371718+0000","last_peered":"2024-09-10T20:59:48.371718+0000","last_clean":"2024-09-10T20:59:48.371718+0000","last_became_active":"2024-09-10T20:59:46.012251+0000","last_became_peered":"2024-09-10T20:59:46.012251+0000","last_unstale":"2024-09-10T20:59:48.371718+0000","last_undegraded":"2024-09-10T20:59:48.371718+0000","last_fullsized":"2024-09-10T20:59:48.371718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:06:09.764168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008181+0000","last_change":"2024-09-10T20:59:47.027750+0000","last_active":"2024-09-10T20:59:48.008181+0000","last_peered":"2024-09-10T20:59:48.008181+0000","last_clean":"2024-09-10T20:59:48.008181+0000","last_became_active":"2024-09-10T20:59:47.027454+0000","last_became_peered":"2024-09-10T20:59:47.027454+0000","last_unstale":"2024-09-10T20:59:48.008181+0000","last_undegraded":"2024-09-10T20:59:48.008181+0000","last_fullsized":"2024-09-10T20:59:48.008181+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:48.206623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.994818+0000","last_change":"2024-09-10T20:59:44.994818+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.994818+0000","last_undegraded":"2024-09-10T20:59:44.994818+0000","last_fullsized":"2024-09-10T20:59:44.994818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024638+0000","last_change":"2024-09-10T20:59:47.030452+0000","last_active":"2024-09-10T20:59:48.024638+0000","last_peered":"2024-09-10T20:59:48.024638+0000","last_clean":"2024-09-10T20:59:48.024638+0000","last_became_active":"2024-09-10T20:59:47.030133+0000","last_became_peered":"2024-09-10T20:59:47.030133+0000","last_unstale":"2024-09-10T20:59:48.024638+0000","last_undegraded":"2024-09-10T20:59:48.024638+0000","last_fullsized":"2024-09-10T20:59:48.024638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:56.952358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008556+0000","last_change":"2024-09-10T20:59:46.014818+0000","last_active":"2024-09-10T20:59:48.008556+0000","last_peered":"2024-09-10T20:59:48.008556+0000","last_clean":"2024-09-10T20:59:48.008556+0000","last_became_active":"2024-09-10T20:59:46.014578+0000","last_became_peered":"2024-09-10T20:59:46.014578+0000","last_unstale":"2024-09-10T20:59:48.008556+0000","last_undegraded":"2024-09-10T20:59:48.008556+0000","last_fullsized":"2024-09-10T20:59:48.008556+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:30.078404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010561+0000","last_change":"2024-09-10T20:59:46.007567+0000","last_active":"2024-09-10T20:59:48.010561+0000","last_peered":"2024-09-10T20:59:48.010561+0000","last_clean":"2024-09-10T20:59:48.010561+0000","last_became_active":"2024-09-10T20:59:46.006773+0000","last_became_peered":"2024-09-10T20:59:46.006773+0000","last_unstale":"2024-09-10T20:59:48.010561+0000","last_undegraded":"2024-09-10T20:59:48.010561+0000","last_fullsized":"2024-09-10T20:59:48.010561+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:19.491620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371345+0000","last_change":"2024-09-10T20:59:47.027217+0000","last_active":"2024-09-10T20:59:48.371345+0000","last_peered":"2024-09-10T20:59:48.371345+0000","last_clean":"2024-09-10T20:59:48.371345+0000","last_became_active":"2024-09-10T20:59:47.026939+0000","last_became_peered":"2024-09-10T20:59:47.026939+0000","last_unstale":"2024-09-10T20:59:48.371345+0000","last_undegraded":"2024-09-10T20:59:48.371345+0000","last_fullsized":"2024-09-10T20:59:48.371345+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:25.592848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009689+0000","last_change":"2024-09-10T20:59:46.010441+0000","last_active":"2024-09-10T20:59:48.009689+0000","last_peered":"2024-09-10T20:59:48.009689+0000","last_clean":"2024-09-10T20:59:48.009689+0000","last_became_active":"2024-09-10T20:59:46.010052+0000","last_became_peered":"2024-09-10T20:59:46.010052+0000","last_unstale":"2024-09-10T20:59:48.009689+0000","last_undegraded":"2024-09-10T20:59:48.009689+0000","last_fullsized":"2024-09-10T20:59:48.009689+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:19.191818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008249+0000","last_change":"2024-09-10T20:59:46.008318+0000","last_active":"2024-09-10T20:59:48.008249+0000","last_peered":"2024-09-10T20:59:48.008249+0000","last_clean":"2024-09-10T20:59:48.008249+0000","last_became_active":"2024-09-10T20:59:46.008069+0000","last_became_peered":"2024-09-10T20:59:46.008069+0000","last_unstale":"2024-09-10T20:59:48.008249+0000","last_undegraded":"2024-09-10T20:59:48.008249+0000","last_fullsized":"2024-09-10T20:59:48.008249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:51.723589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371531+0000","last_change":"2024-09-10T20:59:47.028617+0000","last_active":"2024-09-10T20:59:48.371531+0000","last_peered":"2024-09-10T20:59:48.371531+0000","last_clean":"2024-09-10T20:59:48.371531+0000","last_became_active":"2024-09-10T20:59:47.028282+0000","last_became_peered":"2024-09-10T20:59:47.028282+0000","last_unstale":"2024-09-10T20:59:48.371531+0000","last_undegraded":"2024-09-10T20:59:48.371531+0000","last_fullsized":"2024-09-10T20:59:48.371531+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:35.832692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008386+0000","last_change":"2024-09-10T20:59:46.016007+0000","last_active":"2024-09-10T20:59:48.008386+0000","last_peered":"2024-09-10T20:59:48.008386+0000","last_clean":"2024-09-10T20:59:48.008386+0000","last_became_active":"2024-09-10T20:59:46.015529+0000","last_became_peered":"2024-09-10T20:59:46.015529+0000","last_unstale":"2024-09-10T20:59:48.008386+0000","last_undegraded":"2024-09-10T20:59:48.008386+0000","last_fullsized":"2024-09-10T20:59:48.008386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:58.373425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008159+0000","last_change":"2024-09-10T20:59:47.018746+0000","last_active":"2024-09-10T20:59:48.008159+0000","last_peered":"2024-09-10T20:59:48.008159+0000","last_clean":"2024-09-10T20:59:48.008159+0000","last_became_active":"2024-09-10T20:59:47.018482+0000","last_became_peered":"2024-09-10T20:59:47.018482+0000","last_unstale":"2024-09-10T20:59:48.008159+0000","last_undegraded":"2024-09-10T20:59:48.008159+0000","last_fullsized":"2024-09-10T20:59:48.008159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:17.605108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371407+0000","last_change":"2024-09-10T20:59:46.013441+0000","last_active":"2024-09-10T20:59:48.371407+0000","last_peered":"2024-09-10T20:59:48.371407+0000","last_clean":"2024-09-10T20:59:48.371407+0000","last_became_active":"2024-09-10T20:59:46.012202+0000","last_became_peered":"2024-09-10T20:59:46.012202+0000","last_unstale":"2024-09-10T20:59:48.371407+0000","last_undegraded":"2024-09-10T20:59:48.371407+0000","last_fullsized":"2024-09-10T20:59:48.371407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:59.817251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.997501+0000","last_change":"2024-09-10T20:59:44.997501+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.997501+0000","last_undegraded":"2024-09-10T20:59:44.997501+0000","last_fullsized":"2024-09-10T20:59:44.997501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008204+0000","last_change":"2024-09-10T20:59:47.018576+0000","last_active":"2024-09-10T20:59:48.008204+0000","last_peered":"2024-09-10T20:59:48.008204+0000","last_clean":"2024-09-10T20:59:48.008204+0000","last_became_active":"2024-09-10T20:59:47.018285+0000","last_became_peered":"2024-09-10T20:59:47.018285+0000","last_unstale":"2024-09-10T20:59:48.008204+0000","last_undegraded":"2024-09-10T20:59:48.008204+0000","last_fullsized":"2024-09-10T20:59:48.008204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:32.917475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.998837+0000","last_change":"2024-09-10T20:59:44.998837+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.998837+0000","last_undegraded":"2024-09-10T20:59:44.998837+0000","last_fullsized":"2024-09-10T20:59:44.998837+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008333+0000","last_change":"2024-09-10T20:59:47.037762+0000","last_active":"2024-09-10T20:59:48.008333+0000","last_peered":"2024-09-10T20:59:48.008333+0000","last_clean":"2024-09-10T20:59:48.008333+0000","last_became_active":"2024-09-10T20:59:47.037007+0000","last_became_peered":"2024-09-10T20:59:47.037007+0000","last_unstale":"2024-09-10T20:59:48.008333+0000","last_undegraded":"2024-09-10T20:59:48.008333+0000","last_fullsized":"2024-09-10T20:59:48.008333+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:03.943638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010092+0000","last_change":"2024-09-10T20:59:46.009743+0000","last_active":"2024-09-10T20:59:48.010092+0000","last_peered":"2024-09-10T20:59:48.010092+0000","last_clean":"2024-09-10T20:59:48.010092+0000","last_became_active":"2024-09-10T20:59:46.009491+0000","last_became_peered":"2024-09-10T20:59:46.009491+0000","last_unstale":"2024-09-10T20:59:48.010092+0000","last_undegraded":"2024-09-10T20:59:48.010092+0000","last_fullsized":"2024-09-10T20:59:48.010092+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:56.031945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009586+0000","last_change":"2024-09-10T20:59:47.026805+0000","last_active":"2024-09-10T20:59:48.009586+0000","last_peered":"2024-09-10T20:59:48.009586+0000","last_clean":"2024-09-10T20:59:48.009586+0000","last_became_active":"2024-09-10T20:59:47.026545+0000","last_became_peered":"2024-09-10T20:59:47.026545+0000","last_unstale":"2024-09-10T20:59:48.009586+0000","last_undegraded":"2024-09-10T20:59:48.009586+0000","last_fullsized":"2024-09-10T20:59:48.009586+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:47.670364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.995492+0000","last_change":"2024-09-10T20:59:44.995492+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.995492+0000","last_undegraded":"2024-09-10T20:59:44.995492+0000","last_fullsized":"2024-09-10T20:59:44.995492+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024663+0000","last_change":"2024-09-10T20:59:46.010713+0000","last_active":"2024-09-10T20:59:48.024663+0000","last_peered":"2024-09-10T20:59:48.024663+0000","last_clean":"2024-09-10T20:59:48.024663+0000","last_became_active":"2024-09-10T20:59:46.010448+0000","last_became_peered":"2024-09-10T20:59:46.010448+0000","last_unstale":"2024-09-10T20:59:48.024663+0000","last_undegraded":"2024-09-10T20:59:48.024663+0000","last_fullsized":"2024-09-10T20:59:48.024663+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:41:37.633483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024766+0000","last_change":"2024-09-10T20:59:47.018725+0000","last_active":"2024-09-10T20:59:48.024766+0000","last_peered":"2024-09-10T20:59:48.024766+0000","last_clean":"2024-09-10T20:59:48.024766+0000","last_became_active":"2024-09-10T20:59:47.018486+0000","last_became_peered":"2024-09-10T20:59:47.018486+0000","last_unstale":"2024-09-10T20:59:48.024766+0000","last_undegraded":"2024-09-10T20:59:48.024766+0000","last_fullsized":"2024-09-10T20:59:48.024766+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:37.582285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371690+0000","last_change":"2024-09-10T20:59:46.013035+0000","last_active":"2024-09-10T20:59:48.371690+0000","last_peered":"2024-09-10T20:59:48.371690+0000","last_clean":"2024-09-10T20:59:48.371690+0000","last_became_active":"2024-09-10T20:59:46.012519+0000","last_became_peered":"2024-09-10T20:59:46.012519+0000","last_unstale":"2024-09-10T20:59:48.371690+0000","last_undegraded":"2024-09-10T20:59:48.371690+0000","last_fullsized":"2024-09-10T20:59:48.371690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:38.211184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024461+0000","last_change":"2024-09-10T20:59:46.015090+0000","last_active":"2024-09-10T20:59:48.024461+0000","last_peered":"2024-09-10T20:59:48.024461+0000","last_clean":"2024-09-10T20:59:48.024461+0000","last_became_active":"2024-09-10T20:59:46.014824+0000","last_became_peered":"2024-09-10T20:59:46.014824+0000","last_unstale":"2024-09-10T20:59:48.024461+0000","last_undegraded":"2024-09-10T20:59:48.024461+0000","last_fullsized":"2024-09-10T20:59:48.024461+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:44.918062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371633+0000","last_change":"2024-09-10T20:59:47.028376+0000","last_active":"2024-09-10T20:59:48.371633+0000","last_peered":"2024-09-10T20:59:48.371633+0000","last_clean":"2024-09-10T20:59:48.371633+0000","last_became_active":"2024-09-10T20:59:47.027880+0000","last_became_peered":"2024-09-10T20:59:47.027880+0000","last_unstale":"2024-09-10T20:59:48.371633+0000","last_undegraded":"2024-09-10T20:59:48.371633+0000","last_fullsized":"2024-09-10T20:59:48.371633+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:24.095188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024456+0000","last_change":"2024-09-10T20:59:46.011199+0000","last_active":"2024-09-10T20:59:48.024456+0000","last_peered":"2024-09-10T20:59:48.024456+0000","last_clean":"2024-09-10T20:59:48.024456+0000","last_became_active":"2024-09-10T20:59:46.010933+0000","last_became_peered":"2024-09-10T20:59:46.010933+0000","last_unstale":"2024-09-10T20:59:48.024456+0000","last_undegraded":"2024-09-10T20:59:48.024456+0000","last_fullsized":"2024-09-10T20:59:48.024456+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:37.482570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009795+0000","last_change":"2024-09-10T20:59:47.031076+0000","last_active":"2024-09-10T20:59:48.009795+0000","last_peered":"2024-09-10T20:59:48.009795+0000","last_clean":"2024-09-10T20:59:48.009795+0000","last_became_active":"2024-09-10T20:59:47.030753+0000","last_became_peered":"2024-09-10T20:59:47.030753+0000","last_unstale":"2024-09-10T20:59:48.009795+0000","last_undegraded":"2024-09-10T20:59:48.009795+0000","last_fullsized":"2024-09-10T20:59:48.009795+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:43.434063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.998868+0000","last_change":"2024-09-10T20:59:44.998868+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.998868+0000","last_undegraded":"2024-09-10T20:59:44.998868+0000","last_fullsized":"2024-09-10T20:59:44.998868+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008402+0000","last_change":"2024-09-10T20:59:47.016933+0000","last_active":"2024-09-10T20:59:48.008402+0000","last_peered":"2024-09-10T20:59:48.008402+0000","last_clean":"2024-09-10T20:59:48.008402+0000","last_became_active":"2024-09-10T20:59:47.016668+0000","last_became_peered":"2024-09-10T20:59:47.016668+0000","last_unstale":"2024-09-10T20:59:48.008402+0000","last_undegraded":"2024-09-10T20:59:48.008402+0000","last_fullsized":"2024-09-10T20:59:48.008402+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:02.940429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024373+0000","last_change":"2024-09-10T20:59:46.015133+0000","last_active":"2024-09-10T20:59:48.024373+0000","last_peered":"2024-09-10T20:59:48.024373+0000","last_clean":"2024-09-10T20:59:48.024373+0000","last_became_active":"2024-09-10T20:59:46.014855+0000","last_became_peered":"2024-09-10T20:59:46.014855+0000","last_unstale":"2024-09-10T20:59:48.024373+0000","last_undegraded":"2024-09-10T20:59:48.024373+0000","last_fullsized":"2024-09-10T20:59:48.024373+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:16.570644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371608+0000","last_change":"2024-09-10T20:59:47.028767+0000","last_active":"2024-09-10T20:59:48.371608+0000","last_peered":"2024-09-10T20:59:48.371608+0000","last_clean":"2024-09-10T20:59:48.371608+0000","last_became_active":"2024-09-10T20:59:47.028529+0000","last_became_peered":"2024-09-10T20:59:47.028529+0000","last_unstale":"2024-09-10T20:59:48.371608+0000","last_undegraded":"2024-09-10T20:59:48.371608+0000","last_fullsized":"2024-09-10T20:59:48.371608+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:19.645203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010215+0000","last_change":"2024-09-10T20:59:46.006840+0000","last_active":"2024-09-10T20:59:48.010215+0000","last_peered":"2024-09-10T20:59:48.010215+0000","last_clean":"2024-09-10T20:59:48.010215+0000","last_became_active":"2024-09-10T20:59:46.006570+0000","last_became_peered":"2024-09-10T20:59:46.006570+0000","last_unstale":"2024-09-10T20:59:48.010215+0000","last_undegraded":"2024-09-10T20:59:48.010215+0000","last_fullsized":"2024-09-10T20:59:48.010215+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:11.450731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.026242+0000","last_change":"2024-09-10T20:59:46.013179+0000","last_active":"2024-09-10T20:59:48.026242+0000","last_peered":"2024-09-10T20:59:48.026242+0000","last_clean":"2024-09-10T20:59:48.026242+0000","last_became_active":"2024-09-10T20:59:46.012907+0000","last_became_peered":"2024-09-10T20:59:46.012907+0000","last_unstale":"2024-09-10T20:59:48.026242+0000","last_undegraded":"2024-09-10T20:59:48.026242+0000","last_fullsized":"2024-09-10T20:59:48.026242+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:19.289722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008396+0000","last_change":"2024-09-10T20:59:47.034104+0000","last_active":"2024-09-10T20:59:48.008396+0000","last_peered":"2024-09-10T20:59:48.008396+0000","last_clean":"2024-09-10T20:59:48.008396+0000","last_became_active":"2024-09-10T20:59:47.033795+0000","last_became_peered":"2024-09-10T20:59:47.033795+0000","last_unstale":"2024-09-10T20:59:48.008396+0000","last_undegraded":"2024-09-10T20:59:48.008396+0000","last_fullsized":"2024-09-10T20:59:48.008396+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:59.999686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371561+0000","last_change":"2024-09-10T20:59:46.017352+0000","last_active":"2024-09-10T20:59:48.371561+0000","last_peered":"2024-09-10T20:59:48.371561+0000","last_clean":"2024-09-10T20:59:48.371561+0000","last_became_active":"2024-09-10T20:59:46.017073+0000","last_became_peered":"2024-09-10T20:59:46.017073+0000","last_unstale":"2024-09-10T20:59:48.371561+0000","last_undegraded":"2024-09-10T20:59:48.371561+0000","last_fullsized":"2024-09-10T20:59:48.371561+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:37.017258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371532+0000","last_change":"2024-09-10T20:59:47.025517+0000","last_active":"2024-09-10T20:59:48.371532+0000","last_peered":"2024-09-10T20:59:48.371532+0000","last_clean":"2024-09-10T20:59:48.371532+0000","last_became_active":"2024-09-10T20:59:47.025246+0000","last_became_peered":"2024-09-10T20:59:47.025246+0000","last_unstale":"2024-09-10T20:59:48.371532+0000","last_undegraded":"2024-09-10T20:59:48.371532+0000","last_fullsized":"2024-09-10T20:59:48.371532+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:32.064402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371860+0000","last_change":"2024-09-10T20:59:46.015703+0000","last_active":"2024-09-10T20:59:48.371860+0000","last_peered":"2024-09-10T20:59:48.371860+0000","last_clean":"2024-09-10T20:59:48.371860+0000","last_became_active":"2024-09-10T20:59:46.015257+0000","last_became_peered":"2024-09-10T20:59:46.015257+0000","last_unstale":"2024-09-10T20:59:48.371860+0000","last_undegraded":"2024-09-10T20:59:48.371860+0000","last_fullsized":"2024-09-10T20:59:48.371860+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:24:41.884818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371619+0000","last_change":"2024-09-10T20:59:47.031083+0000","last_active":"2024-09-10T20:59:48.371619+0000","last_peered":"2024-09-10T20:59:48.371619+0000","last_clean":"2024-09-10T20:59:48.371619+0000","last_became_active":"2024-09-10T20:59:47.030792+0000","last_became_peered":"2024-09-10T20:59:47.030792+0000","last_unstale":"2024-09-10T20:59:48.371619+0000","last_undegraded":"2024-09-10T20:59:48.371619+0000","last_fullsized":"2024-09-10T20:59:48.371619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:56.934193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.997504+0000","last_change":"2024-09-10T20:59:44.997504+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.997504+0000","last_undegraded":"2024-09-10T20:59:44.997504+0000","last_fullsized":"2024-09-10T20:59:44.997504+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.995527+0000","last_change":"2024-09-10T20:59:44.995527+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.995527+0000","last_undegraded":"2024-09-10T20:59:44.995527+0000","last_fullsized":"2024-09-10T20:59:44.995527+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T20:59:45.991434+0000","last_change":"2024-09-10T20:59:45.991434+0000","last_active":"2024-09-10T20:59:45.991434+0000","last_peered":"2024-09-10T20:59:45.991434+0000","last_clean":"2024-09-10T20:59:45.991434+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:45.991434+0000","last_undegraded":"2024-09-10T20:59:45.991434+0000","last_fullsized":"2024-09-10T20:59:45.991434+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008062+0000","last_change":"2024-09-10T20:59:46.012810+0000","last_active":"2024-09-10T20:59:48.008062+0000","last_peered":"2024-09-10T20:59:48.008062+0000","last_clean":"2024-09-10T20:59:48.008062+0000","last_became_active":"2024-09-10T20:59:46.012532+0000","last_became_peered":"2024-09-10T20:59:46.012532+0000","last_unstale":"2024-09-10T20:59:48.008062+0000","last_undegraded":"2024-09-10T20:59:48.008062+0000","last_fullsized":"2024-09-10T20:59:48.008062+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:25.702550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009911+0000","last_change":"2024-09-10T20:59:47.032222+0000","last_active":"2024-09-10T20:59:48.009911+0000","last_peered":"2024-09-10T20:59:48.009911+0000","last_clean":"2024-09-10T20:59:48.009911+0000","last_became_active":"2024-09-10T20:59:47.031463+0000","last_became_peered":"2024-09-10T20:59:47.031463+0000","last_unstale":"2024-09-10T20:59:48.009911+0000","last_undegraded":"2024-09-10T20:59:48.009911+0000","last_fullsized":"2024-09-10T20:59:48.009911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:24.386660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-10T20:59:44.998797+0000","last_change":"2024-09-10T20:59:44.998797+0000","last_active":"2024-09-10T20:59:44.986452+0000","last_peered":"2024-09-10T20:59:44.986452+0000","last_clean":"2024-09-10T20:59:44.986452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T20:59:44.998797+0000","last_undegraded":"2024-09-10T20:59:44.998797+0000","last_fullsized":"2024-09-10T20:59:44.998797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":140,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T20:59:44.999128+0000","last_change":"2024-09-10T20:58:36.895993+0000","last_active":"2024-09-10T20:59:44.999128+0000","last_peered":"2024-09-10T20:59:44.999128+0000","last_clean":"2024-09-10T20:59:44.999128+0000","last_became_active":"2024-09-10T20:58:36.894988+0000","last_became_peered":"2024-09-10T20:58:36.894988+0000","last_unstale":"2024-09-10T20:59:44.999128+0000","last_undegraded":"2024-09-10T20:59:44.999128+0000","last_fullsized":"2024-09-10T20:59:44.999128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_clean_scrub_stamp":"2024-09-10T20:58:35.588702+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.082528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":82,"acting":82,"num_store_stats":5},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":767894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":177526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":177526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":471040,"data_stored":199698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":198122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":198142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1052672,"data_stored":788454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":198164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:59:54.695 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T20:59:54.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.806+0000 7f28adf45640 1 -- 172.21.15.98:0/1129436975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28a80d2830 msgr2=0x7f28a80cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:54.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.806+0000 7f28adf45640 1 --2- 172.21.15.98:0/1129436975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28a80d2830 0x7f28a80cf8e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f28a8073af0 tx=0x7f28a002a9e0 comp rx=0 tx=0).stop 2024-09-10T20:59:54.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.807+0000 7f28adf45640 1 -- 172.21.15.98:0/1129436975 shutdown_connections 2024-09-10T20:59:54.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.807+0000 7f28adf45640 1 --2- 172.21.15.98:0/1129436975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a80d0a00 0x7f28a8070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:54.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.807+0000 7f28adf45640 1 --2- 172.21.15.98:0/1129436975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28a80d2830 0x7f28a80cf8e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:54.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.807+0000 7f28adf45640 1 -- 172.21.15.98:0/1129436975 >> 172.21.15.98:0/1129436975 conn(0x7f28a80d1d10 msgr2=0x7f28a80d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:54.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.807+0000 7f28adf45640 1 -- 172.21.15.98:0/1129436975 shutdown_connections 2024-09-10T20:59:54.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.807+0000 7f28adf45640 1 -- 172.21.15.98:0/1129436975 wait complete. 2024-09-10T20:59:54.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.808+0000 7f28adf45640 1 Processor -- start 2024-09-10T20:59:54.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.809+0000 7f28adf45640 1 -- start start 2024-09-10T20:59:54.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.810+0000 7f28adf45640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28a80d0a00 0x7f28a81f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:54.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.810+0000 7f28adf45640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a80d2830 0x7f28a80b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:54.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.810+0000 7f28adf45640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28a815be00 0x7f28a815e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:54.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.810+0000 7f28adf45640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f28a80cd1d0 con 0x7f28a80d0a00 2024-09-10T20:59:54.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.810+0000 7f28adf45640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f28a80cd050 con 0x7f28a815be00 2024-09-10T20:59:54.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.810+0000 7f28adf45640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f28a80cd350 con 0x7f28a80d2830 2024-09-10T20:59:54.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.810+0000 7f28a77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28a80d0a00 0x7f28a81f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:54.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.810+0000 7f28a77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28a80d0a00 0x7f28a81f5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54884/0 (socket says 172.21.15.98:54884) 2024-09-10T20:59:54.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.810+0000 7f28a77fe640 1 -- 172.21.15.98:0/1799173552 learned_addr learned my addr 172.21.15.98:0/1799173552 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:54.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.810+0000 7f28a7fff640 1 --2- 172.21.15.98:0/1799173552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28a815be00 0x7f28a815e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:54.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.810+0000 7f28a6ffd640 1 --2- 172.21.15.98:0/1799173552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a80d2830 0x7f28a80b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:54.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.810+0000 7f28a77fe640 1 -- 172.21.15.98:0/1799173552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a80d2830 msgr2=0x7f28a80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:54.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.810+0000 7f28a77fe640 1 --2- 172.21.15.98:0/1799173552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a80d2830 0x7f28a80b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:54.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.811+0000 7f28a77fe640 1 -- 172.21.15.98:0/1799173552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28a815be00 msgr2=0x7f28a815e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:54.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.811+0000 7f28a77fe640 1 --2- 172.21.15.98:0/1799173552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28a815be00 0x7f28a815e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:54.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.811+0000 7f28a77fe640 1 -- 172.21.15.98:0/1799173552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a80c9e90 con 0x7f28a80d0a00 2024-09-10T20:59:54.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.811+0000 7f28a77fe640 1 --2- 172.21.15.98:0/1799173552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28a80d0a00 0x7f28a81f5f40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f289800b3f0 tx=0x7f289800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:54.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.811+0000 7f289ffff640 1 -- 172.21.15.98:0/1799173552 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898015020 con 0x7f28a80d0a00 2024-09-10T20:59:54.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.811+0000 7f289ffff640 1 -- 172.21.15.98:0/1799173552 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2898011920 con 0x7f28a80d0a00 2024-09-10T20:59:54.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.811+0000 7f28adf45640 1 -- 172.21.15.98:0/1799173552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a80b5fa0 con 0x7f28a80d0a00 2024-09-10T20:59:54.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.811+0000 7f289ffff640 1 -- 172.21.15.98:0/1799173552 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898011d70 con 0x7f28a80d0a00 2024-09-10T20:59:54.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.811+0000 7f28adf45640 1 -- 172.21.15.98:0/1799173552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a80ca3f0 con 0x7f28a80d0a00 2024-09-10T20:59:54.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.812+0000 7f28adf45640 1 -- 172.21.15.98:0/1799173552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28a8070760 con 0x7f28a80d0a00 2024-09-10T20:59:54.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.813+0000 7f289ffff640 1 -- 172.21.15.98:0/1799173552 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f289801a400 con 0x7f28a80d0a00 2024-09-10T20:59:54.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.813+0000 7f289ffff640 1 --2- 172.21.15.98:0/1799173552 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2888077f70 0x7f288807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:54.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.814+0000 7f289ffff640 1 -- 172.21.15.98:0/1799173552 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f2898099de0 con 0x7f28a80d0a00 2024-09-10T20:59:54.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.814+0000 7f28a6ffd640 1 --2- 172.21.15.98:0/1799173552 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2888077f70 0x7f288807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:54.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.814+0000 7f28a6ffd640 1 --2- 172.21.15.98:0/1799173552 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2888077f70 0x7f288807a430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2890005fd0 tx=0x7f2890005e20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:54.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.817+0000 7f289ffff640 1 -- 172.21.15.98:0/1799173552 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f289801f030 con 0x7f28a80d0a00 2024-09-10T20:59:55.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:54.997+0000 7f28adf45640 1 -- 172.21.15.98:0/1799173552 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28a80b63f0 con 0x7f2888077f70 2024-09-10T20:59:55.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.002+0000 7f289ffff640 1 -- 172.21.15.98:0/1799173552 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7f28a80b63f0 con 0x7f2888077f70 2024-09-10T20:59:55.010 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:55.012 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-10T20:59:55.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.005+0000 7f28adf45640 1 -- 172.21.15.98:0/1799173552 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2888077f70 msgr2=0x7f288807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.005+0000 7f28adf45640 1 --2- 172.21.15.98:0/1799173552 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2888077f70 0x7f288807a430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2890005fd0 tx=0x7f2890005e20 comp rx=0 tx=0).stop 2024-09-10T20:59:55.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.005+0000 7f28adf45640 1 -- 172.21.15.98:0/1799173552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28a80d0a00 msgr2=0x7f28a81f5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.005+0000 7f28adf45640 1 --2- 172.21.15.98:0/1799173552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28a80d0a00 0x7f28a81f5f40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f289800b3f0 tx=0x7f289800b8b0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.005+0000 7f28adf45640 1 -- 172.21.15.98:0/1799173552 shutdown_connections 2024-09-10T20:59:55.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.005+0000 7f28adf45640 1 --2- 172.21.15.98:0/1799173552 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2888077f70 0x7f288807a430 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.005+0000 7f28adf45640 1 --2- 172.21.15.98:0/1799173552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28a815be00 0x7f28a815e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.005+0000 7f28adf45640 1 --2- 172.21.15.98:0/1799173552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f28a80d2830 0x7f28a80b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.005+0000 7f28adf45640 1 --2- 172.21.15.98:0/1799173552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f28a80d0a00 0x7f28a81f5f40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.005+0000 7f28adf45640 1 -- 172.21.15.98:0/1799173552 >> 172.21.15.98:0/1799173552 conn(0x7f28a80d1d10 msgr2=0x7f28a80aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:55.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.005+0000 7f28adf45640 1 -- 172.21.15.98:0/1799173552 shutdown_connections 2024-09-10T20:59:55.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.005+0000 7f28adf45640 1 -- 172.21.15.98:0/1799173552 wait complete. 2024-09-10T20:59:55.021 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-09-10T20:59:54.922643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221020,"kb_used_data":4776,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753700,"statfs":{"total":773094113280,"available":772867788800,"internally_reserved":0,"allocated":4890624,"data_stored":2769972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002179"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008495+0000","last_change":"2024-09-10T20:59:47.020403+0000","last_active":"2024-09-10T20:59:48.008495+0000","last_peered":"2024-09-10T20:59:48.008495+0000","last_clean":"2024-09-10T20:59:48.008495+0000","last_became_active":"2024-09-10T20:59:47.018947+0000","last_became_peered":"2024-09-10T20:59:47.018947+0000","last_unstale":"2024-09-10T20:59:48.008495+0000","last_undegraded":"2024-09-10T20:59:48.008495+0000","last_fullsized":"2024-09-10T20:59:48.008495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:41.452362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008383+0000","last_change":"2024-09-10T20:59:47.022856+0000","last_active":"2024-09-10T20:59:48.008383+0000","last_peered":"2024-09-10T20:59:48.008383+0000","last_clean":"2024-09-10T20:59:48.008383+0000","last_became_active":"2024-09-10T20:59:47.022579+0000","last_became_peered":"2024-09-10T20:59:47.022579+0000","last_unstale":"2024-09-10T20:59:48.008383+0000","last_undegraded":"2024-09-10T20:59:48.008383+0000","last_fullsized":"2024-09-10T20:59:48.008383+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:51.434814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371766+0000","last_change":"2024-09-10T20:59:47.028091+0000","last_active":"2024-09-10T20:59:48.371766+0000","last_peered":"2024-09-10T20:59:48.371766+0000","last_clean":"2024-09-10T20:59:48.371766+0000","last_became_active":"2024-09-10T20:59:47.027830+0000","last_became_peered":"2024-09-10T20:59:47.027830+0000","last_unstale":"2024-09-10T20:59:48.371766+0000","last_undegraded":"2024-09-10T20:59:48.371766+0000","last_fullsized":"2024-09-10T20:59:48.371766+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:17.875896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024599+0000","last_change":"2024-09-10T20:59:47.033657+0000","last_active":"2024-09-10T20:59:48.024599+0000","last_peered":"2024-09-10T20:59:48.024599+0000","last_clean":"2024-09-10T20:59:48.024599+0000","last_became_active":"2024-09-10T20:59:47.033363+0000","last_became_peered":"2024-09-10T20:59:47.033363+0000","last_unstale":"2024-09-10T20:59:48.024599+0000","last_undegraded":"2024-09-10T20:59:48.024599+0000","last_fullsized":"2024-09-10T20:59:48.024599+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:45.439857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024571+0000","last_change":"2024-09-10T20:59:47.030433+0000","last_active":"2024-09-10T20:59:48.024571+0000","last_peered":"2024-09-10T20:59:48.024571+0000","last_clean":"2024-09-10T20:59:48.024571+0000","last_became_active":"2024-09-10T20:59:47.030093+0000","last_became_peered":"2024-09-10T20:59:47.030093+0000","last_unstale":"2024-09-10T20:59:48.024571+0000","last_undegraded":"2024-09-10T20:59:48.024571+0000","last_fullsized":"2024-09-10T20:59:48.024571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:23.601594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371305+0000","last_change":"2024-09-10T20:59:47.022075+0000","last_active":"2024-09-10T20:59:48.371305+0000","last_peered":"2024-09-10T20:59:48.371305+0000","last_clean":"2024-09-10T20:59:48.371305+0000","last_became_active":"2024-09-10T20:59:47.021810+0000","last_became_peered":"2024-09-10T20:59:47.021810+0000","last_unstale":"2024-09-10T20:59:48.371305+0000","last_undegraded":"2024-09-10T20:59:48.371305+0000","last_fullsized":"2024-09-10T20:59:48.371305+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:53.763319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009198+0000","last_change":"2024-09-10T20:59:47.024763+0000","last_active":"2024-09-10T20:59:48.009198+0000","last_peered":"2024-09-10T20:59:48.009198+0000","last_clean":"2024-09-10T20:59:48.009198+0000","last_became_active":"2024-09-10T20:59:47.024475+0000","last_became_peered":"2024-09-10T20:59:47.024475+0000","last_unstale":"2024-09-10T20:59:48.009198+0000","last_undegraded":"2024-09-10T20:59:48.009198+0000","last_fullsized":"2024-09-10T20:59:48.009198+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:51:57.920889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008071+0000","last_change":"2024-09-10T20:59:47.026033+0000","last_active":"2024-09-10T20:59:48.008071+0000","last_peered":"2024-09-10T20:59:48.008071+0000","last_clean":"2024-09-10T20:59:48.008071+0000","last_became_active":"2024-09-10T20:59:47.025434+0000","last_became_peered":"2024-09-10T20:59:47.025434+0000","last_unstale":"2024-09-10T20:59:48.008071+0000","last_undegraded":"2024-09-10T20:59:48.008071+0000","last_fullsized":"2024-09-10T20:59:48.008071+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:28.753205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371967+0000","last_change":"2024-09-10T20:59:47.028797+0000","last_active":"2024-09-10T20:59:48.371967+0000","last_peered":"2024-09-10T20:59:48.371967+0000","last_clean":"2024-09-10T20:59:48.371967+0000","last_became_active":"2024-09-10T20:59:47.028522+0000","last_became_peered":"2024-09-10T20:59:47.028522+0000","last_unstale":"2024-09-10T20:59:48.371967+0000","last_undegraded":"2024-09-10T20:59:48.371967+0000","last_fullsized":"2024-09-10T20:59:48.371967+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:35.047921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010611+0000","last_change":"2024-09-10T20:59:47.028365+0000","last_active":"2024-09-10T20:59:48.010611+0000","last_peered":"2024-09-10T20:59:48.010611+0000","last_clean":"2024-09-10T20:59:48.010611+0000","last_became_active":"2024-09-10T20:59:47.028064+0000","last_became_peered":"2024-09-10T20:59:47.028064+0000","last_unstale":"2024-09-10T20:59:48.010611+0000","last_undegraded":"2024-09-10T20:59:48.010611+0000","last_fullsized":"2024-09-10T20:59:48.010611+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:39.938414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009375+0000","last_change":"2024-09-10T20:59:47.031189+0000","last_active":"2024-09-10T20:59:48.009375+0000","last_peered":"2024-09-10T20:59:48.009375+0000","last_clean":"2024-09-10T20:59:48.009375+0000","last_became_active":"2024-09-10T20:59:47.030852+0000","last_became_peered":"2024-09-10T20:59:47.030852+0000","last_unstale":"2024-09-10T20:59:48.009375+0000","last_undegraded":"2024-09-10T20:59:48.009375+0000","last_fullsized":"2024-09-10T20:59:48.009375+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:49.659631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371334+0000","last_change":"2024-09-10T20:59:47.024031+0000","last_active":"2024-09-10T20:59:48.371334+0000","last_peered":"2024-09-10T20:59:48.371334+0000","last_clean":"2024-09-10T20:59:48.371334+0000","last_became_active":"2024-09-10T20:59:47.023695+0000","last_became_peered":"2024-09-10T20:59:47.023695+0000","last_unstale":"2024-09-10T20:59:48.371334+0000","last_undegraded":"2024-09-10T20:59:48.371334+0000","last_fullsized":"2024-09-10T20:59:48.371334+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:09.439607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009786+0000","last_change":"2024-09-10T20:59:47.028762+0000","last_active":"2024-09-10T20:59:48.009786+0000","last_peered":"2024-09-10T20:59:48.009786+0000","last_clean":"2024-09-10T20:59:48.009786+0000","last_became_active":"2024-09-10T20:59:47.028406+0000","last_became_peered":"2024-09-10T20:59:47.028406+0000","last_unstale":"2024-09-10T20:59:48.009786+0000","last_undegraded":"2024-09-10T20:59:48.009786+0000","last_fullsized":"2024-09-10T20:59:48.009786+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:12.795591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008997+0000","last_change":"2024-09-10T20:59:47.033553+0000","last_active":"2024-09-10T20:59:48.008997+0000","last_peered":"2024-09-10T20:59:48.008997+0000","last_clean":"2024-09-10T20:59:48.008997+0000","last_became_active":"2024-09-10T20:59:47.033292+0000","last_became_peered":"2024-09-10T20:59:47.033292+0000","last_unstale":"2024-09-10T20:59:48.008997+0000","last_undegraded":"2024-09-10T20:59:48.008997+0000","last_fullsized":"2024-09-10T20:59:48.008997+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:30.038046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009388+0000","last_change":"2024-09-10T20:59:47.024046+0000","last_active":"2024-09-10T20:59:48.009388+0000","last_peered":"2024-09-10T20:59:48.009388+0000","last_clean":"2024-09-10T20:59:48.009388+0000","last_became_active":"2024-09-10T20:59:47.023745+0000","last_became_peered":"2024-09-10T20:59:47.023745+0000","last_unstale":"2024-09-10T20:59:48.009388+0000","last_undegraded":"2024-09-10T20:59:48.009388+0000","last_fullsized":"2024-09-10T20:59:48.009388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:32.861565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371635+0000","last_change":"2024-09-10T20:59:47.028025+0000","last_active":"2024-09-10T20:59:48.371635+0000","last_peered":"2024-09-10T20:59:48.371635+0000","last_clean":"2024-09-10T20:59:48.371635+0000","last_became_active":"2024-09-10T20:59:47.027065+0000","last_became_peered":"2024-09-10T20:59:47.027065+0000","last_unstale":"2024-09-10T20:59:48.371635+0000","last_undegraded":"2024-09-10T20:59:48.371635+0000","last_fullsized":"2024-09-10T20:59:48.371635+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:17.875896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.372645+0000","last_change":"2024-09-10T20:59:47.027453+0000","last_active":"2024-09-10T20:59:48.372645+0000","last_peered":"2024-09-10T20:59:48.372645+0000","last_clean":"2024-09-10T20:59:48.372645+0000","last_became_active":"2024-09-10T20:59:47.027130+0000","last_became_peered":"2024-09-10T20:59:47.027130+0000","last_unstale":"2024-09-10T20:59:48.372645+0000","last_undegraded":"2024-09-10T20:59:48.372645+0000","last_fullsized":"2024-09-10T20:59:48.372645+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:15.373447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.372562+0000","last_change":"2024-09-10T20:59:47.036000+0000","last_active":"2024-09-10T20:59:48.372562+0000","last_peered":"2024-09-10T20:59:48.372562+0000","last_clean":"2024-09-10T20:59:48.372562+0000","last_became_active":"2024-09-10T20:59:47.035703+0000","last_became_peered":"2024-09-10T20:59:47.035703+0000","last_unstale":"2024-09-10T20:59:48.372562+0000","last_undegraded":"2024-09-10T20:59:48.372562+0000","last_fullsized":"2024-09-10T20:59:48.372562+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:41.166136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024695+0000","last_change":"2024-09-10T20:59:47.030399+0000","last_active":"2024-09-10T20:59:48.024695+0000","last_peered":"2024-09-10T20:59:48.024695+0000","last_clean":"2024-09-10T20:59:48.024695+0000","last_became_active":"2024-09-10T20:59:47.029699+0000","last_became_peered":"2024-09-10T20:59:47.029699+0000","last_unstale":"2024-09-10T20:59:48.024695+0000","last_undegraded":"2024-09-10T20:59:48.024695+0000","last_fullsized":"2024-09-10T20:59:48.024695+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:26.745564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.372147+0000","last_change":"2024-09-10T20:59:47.035863+0000","last_active":"2024-09-10T20:59:48.372147+0000","last_peered":"2024-09-10T20:59:48.372147+0000","last_clean":"2024-09-10T20:59:48.372147+0000","last_became_active":"2024-09-10T20:59:47.035606+0000","last_became_peered":"2024-09-10T20:59:47.035606+0000","last_unstale":"2024-09-10T20:59:48.372147+0000","last_undegraded":"2024-09-10T20:59:48.372147+0000","last_fullsized":"2024-09-10T20:59:48.372147+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:44.085620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009982+0000","last_change":"2024-09-10T20:59:47.028032+0000","last_active":"2024-09-10T20:59:48.009982+0000","last_peered":"2024-09-10T20:59:48.009982+0000","last_clean":"2024-09-10T20:59:48.009982+0000","last_became_active":"2024-09-10T20:59:47.027708+0000","last_became_peered":"2024-09-10T20:59:47.027708+0000","last_unstale":"2024-09-10T20:59:48.009982+0000","last_undegraded":"2024-09-10T20:59:48.009982+0000","last_fullsized":"2024-09-10T20:59:48.009982+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:48.185509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009218+0000","last_change":"2024-09-10T20:59:47.030477+0000","last_active":"2024-09-10T20:59:48.009218+0000","last_peered":"2024-09-10T20:59:48.009218+0000","last_clean":"2024-09-10T20:59:48.009218+0000","last_became_active":"2024-09-10T20:59:47.030217+0000","last_became_peered":"2024-09-10T20:59:47.030217+0000","last_unstale":"2024-09-10T20:59:48.009218+0000","last_undegraded":"2024-09-10T20:59:48.009218+0000","last_fullsized":"2024-09-10T20:59:48.009218+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:41.166136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371983+0000","last_change":"2024-09-10T20:59:47.022119+0000","last_active":"2024-09-10T20:59:48.371983+0000","last_peered":"2024-09-10T20:59:48.371983+0000","last_clean":"2024-09-10T20:59:48.371983+0000","last_became_active":"2024-09-10T20:59:47.021836+0000","last_became_peered":"2024-09-10T20:59:47.021836+0000","last_unstale":"2024-09-10T20:59:48.371983+0000","last_undegraded":"2024-09-10T20:59:48.371983+0000","last_fullsized":"2024-09-10T20:59:48.371983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:19.023203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024474+0000","last_change":"2024-09-10T20:59:47.027243+0000","last_active":"2024-09-10T20:59:48.024474+0000","last_peered":"2024-09-10T20:59:48.024474+0000","last_clean":"2024-09-10T20:59:48.024474+0000","last_became_active":"2024-09-10T20:59:47.026976+0000","last_became_peered":"2024-09-10T20:59:47.026976+0000","last_unstale":"2024-09-10T20:59:48.024474+0000","last_undegraded":"2024-09-10T20:59:48.024474+0000","last_fullsized":"2024-09-10T20:59:48.024474+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:17.556094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.372434+0000","last_change":"2024-09-10T20:59:47.036488+0000","last_active":"2024-09-10T20:59:48.372434+0000","last_peered":"2024-09-10T20:59:48.372434+0000","last_clean":"2024-09-10T20:59:48.372434+0000","last_became_active":"2024-09-10T20:59:47.036233+0000","last_became_peered":"2024-09-10T20:59:47.036233+0000","last_unstale":"2024-09-10T20:59:48.372434+0000","last_undegraded":"2024-09-10T20:59:48.372434+0000","last_fullsized":"2024-09-10T20:59:48.372434+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:30.038046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371446+0000","last_change":"2024-09-10T20:59:47.029424+0000","last_active":"2024-09-10T20:59:48.371446+0000","last_peered":"2024-09-10T20:59:48.371446+0000","last_clean":"2024-09-10T20:59:48.371446+0000","last_became_active":"2024-09-10T20:59:47.029180+0000","last_became_peered":"2024-09-10T20:59:47.029180+0000","last_unstale":"2024-09-10T20:59:48.371446+0000","last_undegraded":"2024-09-10T20:59:48.371446+0000","last_fullsized":"2024-09-10T20:59:48.371446+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:31.373458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.372519+0000","last_change":"2024-09-10T20:59:47.039494+0000","last_active":"2024-09-10T20:59:48.372519+0000","last_peered":"2024-09-10T20:59:48.372519+0000","last_clean":"2024-09-10T20:59:48.372519+0000","last_became_active":"2024-09-10T20:59:47.039197+0000","last_became_peered":"2024-09-10T20:59:47.039197+0000","last_unstale":"2024-09-10T20:59:48.372519+0000","last_undegraded":"2024-09-10T20:59:48.372519+0000","last_fullsized":"2024-09-10T20:59:48.372519+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:48.223095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010216+0000","last_change":"2024-09-10T20:59:47.025305+0000","last_active":"2024-09-10T20:59:48.010216+0000","last_peered":"2024-09-10T20:59:48.010216+0000","last_clean":"2024-09-10T20:59:48.010216+0000","last_became_active":"2024-09-10T20:59:47.025036+0000","last_became_peered":"2024-09-10T20:59:47.025036+0000","last_unstale":"2024-09-10T20:59:48.010216+0000","last_undegraded":"2024-09-10T20:59:48.010216+0000","last_fullsized":"2024-09-10T20:59:48.010216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:25.337419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.372321+0000","last_change":"2024-09-10T20:59:47.030814+0000","last_active":"2024-09-10T20:59:48.372321+0000","last_peered":"2024-09-10T20:59:48.372321+0000","last_clean":"2024-09-10T20:59:48.372321+0000","last_became_active":"2024-09-10T20:59:47.030546+0000","last_became_peered":"2024-09-10T20:59:47.030546+0000","last_unstale":"2024-09-10T20:59:48.372321+0000","last_undegraded":"2024-09-10T20:59:48.372321+0000","last_fullsized":"2024-09-10T20:59:48.372321+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:56.179687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371959+0000","last_change":"2024-09-10T20:59:47.035154+0000","last_active":"2024-09-10T20:59:48.371959+0000","last_peered":"2024-09-10T20:59:48.371959+0000","last_clean":"2024-09-10T20:59:48.371959+0000","last_became_active":"2024-09-10T20:59:47.034853+0000","last_became_peered":"2024-09-10T20:59:47.034853+0000","last_unstale":"2024-09-10T20:59:48.371959+0000","last_undegraded":"2024-09-10T20:59:48.371959+0000","last_fullsized":"2024-09-10T20:59:48.371959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:37.279273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371990+0000","last_change":"2024-09-10T20:59:47.026644+0000","last_active":"2024-09-10T20:59:48.371990+0000","last_peered":"2024-09-10T20:59:48.371990+0000","last_clean":"2024-09-10T20:59:48.371990+0000","last_became_active":"2024-09-10T20:59:47.026302+0000","last_became_peered":"2024-09-10T20:59:47.026302+0000","last_unstale":"2024-09-10T20:59:48.371990+0000","last_undegraded":"2024-09-10T20:59:48.371990+0000","last_fullsized":"2024-09-10T20:59:48.371990+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:00.339854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371305+0000","last_change":"2024-09-10T20:59:47.021237+0000","last_active":"2024-09-10T20:59:48.371305+0000","last_peered":"2024-09-10T20:59:48.371305+0000","last_clean":"2024-09-10T20:59:48.371305+0000","last_became_active":"2024-09-10T20:59:47.020964+0000","last_became_peered":"2024-09-10T20:59:47.020964+0000","last_unstale":"2024-09-10T20:59:48.371305+0000","last_undegraded":"2024-09-10T20:59:48.371305+0000","last_fullsized":"2024-09-10T20:59:48.371305+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:50.897033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.372602+0000","last_change":"2024-09-10T20:59:47.027016+0000","last_active":"2024-09-10T20:59:48.372602+0000","last_peered":"2024-09-10T20:59:48.372602+0000","last_clean":"2024-09-10T20:59:48.372602+0000","last_became_active":"2024-09-10T20:59:47.026718+0000","last_became_peered":"2024-09-10T20:59:47.026718+0000","last_unstale":"2024-09-10T20:59:48.372602+0000","last_undegraded":"2024-09-10T20:59:48.372602+0000","last_fullsized":"2024-09-10T20:59:48.372602+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:44.706593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.372343+0000","last_change":"2024-09-10T20:59:46.008443+0000","last_active":"2024-09-10T20:59:48.372343+0000","last_peered":"2024-09-10T20:59:48.372343+0000","last_clean":"2024-09-10T20:59:48.372343+0000","last_became_active":"2024-09-10T20:59:46.008225+0000","last_became_peered":"2024-09-10T20:59:46.008225+0000","last_unstale":"2024-09-10T20:59:48.372343+0000","last_undegraded":"2024-09-10T20:59:48.372343+0000","last_fullsized":"2024-09-10T20:59:48.372343+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:51.458006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371836+0000","last_change":"2024-09-10T20:59:47.028678+0000","last_active":"2024-09-10T20:59:48.371836+0000","last_peered":"2024-09-10T20:59:48.371836+0000","last_clean":"2024-09-10T20:59:48.371836+0000","last_became_active":"2024-09-10T20:59:47.028134+0000","last_became_peered":"2024-09-10T20:59:47.028134+0000","last_unstale":"2024-09-10T20:59:48.371836+0000","last_undegraded":"2024-09-10T20:59:48.371836+0000","last_fullsized":"2024-09-10T20:59:48.371836+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:21.297545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371469+0000","last_change":"2024-09-10T20:59:46.014029+0000","last_active":"2024-09-10T20:59:48.371469+0000","last_peered":"2024-09-10T20:59:48.371469+0000","last_clean":"2024-09-10T20:59:48.371469+0000","last_became_active":"2024-09-10T20:59:46.013771+0000","last_became_peered":"2024-09-10T20:59:46.013771+0000","last_unstale":"2024-09-10T20:59:48.371469+0000","last_undegraded":"2024-09-10T20:59:48.371469+0000","last_fullsized":"2024-09-10T20:59:48.371469+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:05.365319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009949+0000","last_change":"2024-09-10T20:59:47.036997+0000","last_active":"2024-09-10T20:59:48.009949+0000","last_peered":"2024-09-10T20:59:48.009949+0000","last_clean":"2024-09-10T20:59:48.009949+0000","last_became_active":"2024-09-10T20:59:47.036699+0000","last_became_peered":"2024-09-10T20:59:47.036699+0000","last_unstale":"2024-09-10T20:59:48.009949+0000","last_undegraded":"2024-09-10T20:59:48.009949+0000","last_fullsized":"2024-09-10T20:59:48.009949+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:35.832692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371865+0000","last_change":"2024-09-10T20:59:46.012293+0000","last_active":"2024-09-10T20:59:48.371865+0000","last_peered":"2024-09-10T20:59:48.371865+0000","last_clean":"2024-09-10T20:59:48.371865+0000","last_became_active":"2024-09-10T20:59:46.012014+0000","last_became_peered":"2024-09-10T20:59:46.012014+0000","last_unstale":"2024-09-10T20:59:48.371865+0000","last_undegraded":"2024-09-10T20:59:48.371865+0000","last_fullsized":"2024-09-10T20:59:48.371865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:30.723075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008466+0000","last_change":"2024-09-10T20:59:47.032715+0000","last_active":"2024-09-10T20:59:48.008466+0000","last_peered":"2024-09-10T20:59:48.008466+0000","last_clean":"2024-09-10T20:59:48.008466+0000","last_became_active":"2024-09-10T20:59:47.032448+0000","last_became_peered":"2024-09-10T20:59:47.032448+0000","last_unstale":"2024-09-10T20:59:48.008466+0000","last_undegraded":"2024-09-10T20:59:48.008466+0000","last_fullsized":"2024-09-10T20:59:48.008466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:34.824871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009179+0000","last_change":"2024-09-10T20:59:46.013615+0000","last_active":"2024-09-10T20:59:48.009179+0000","last_peered":"2024-09-10T20:59:48.009179+0000","last_clean":"2024-09-10T20:59:48.009179+0000","last_became_active":"2024-09-10T20:59:46.013356+0000","last_became_peered":"2024-09-10T20:59:46.013356+0000","last_unstale":"2024-09-10T20:59:48.009179+0000","last_undegraded":"2024-09-10T20:59:48.009179+0000","last_fullsized":"2024-09-10T20:59:48.009179+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:06.650789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371308+0000","last_change":"2024-09-10T20:59:47.026327+0000","last_active":"2024-09-10T20:59:48.371308+0000","last_peered":"2024-09-10T20:59:48.371308+0000","last_clean":"2024-09-10T20:59:48.371308+0000","last_became_active":"2024-09-10T20:59:47.026074+0000","last_became_peered":"2024-09-10T20:59:47.026074+0000","last_unstale":"2024-09-10T20:59:48.371308+0000","last_undegraded":"2024-09-10T20:59:48.371308+0000","last_fullsized":"2024-09-10T20:59:48.371308+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:41.166136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010096+0000","last_change":"2024-09-10T20:59:46.014169+0000","last_active":"2024-09-10T20:59:48.010096+0000","last_peered":"2024-09-10T20:59:48.010096+0000","last_clean":"2024-09-10T20:59:48.010096+0000","last_became_active":"2024-09-10T20:59:46.013940+0000","last_became_peered":"2024-09-10T20:59:46.013940+0000","last_unstale":"2024-09-10T20:59:48.010096+0000","last_undegraded":"2024-09-10T20:59:48.010096+0000","last_fullsized":"2024-09-10T20:59:48.010096+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:52.758336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009576+0000","last_change":"2024-09-10T20:59:47.027620+0000","last_active":"2024-09-10T20:59:48.009576+0000","last_peered":"2024-09-10T20:59:48.009576+0000","last_clean":"2024-09-10T20:59:48.009576+0000","last_became_active":"2024-09-10T20:59:47.027339+0000","last_became_peered":"2024-09-10T20:59:47.027339+0000","last_unstale":"2024-09-10T20:59:48.009576+0000","last_undegraded":"2024-09-10T20:59:48.009576+0000","last_fullsized":"2024-09-10T20:59:48.009576+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:37.031813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371717+0000","last_change":"2024-09-10T20:59:46.008649+0000","last_active":"2024-09-10T20:59:48.371717+0000","last_peered":"2024-09-10T20:59:48.371717+0000","last_clean":"2024-09-10T20:59:48.371717+0000","last_became_active":"2024-09-10T20:59:46.008410+0000","last_became_peered":"2024-09-10T20:59:46.008410+0000","last_unstale":"2024-09-10T20:59:48.371717+0000","last_undegraded":"2024-09-10T20:59:48.371717+0000","last_fullsized":"2024-09-10T20:59:48.371717+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:32.019926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009545+0000","last_change":"2024-09-10T20:59:47.028938+0000","last_active":"2024-09-10T20:59:48.009545+0000","last_peered":"2024-09-10T20:59:48.009545+0000","last_clean":"2024-09-10T20:59:48.009545+0000","last_became_active":"2024-09-10T20:59:47.028604+0000","last_became_peered":"2024-09-10T20:59:47.028604+0000","last_unstale":"2024-09-10T20:59:48.009545+0000","last_undegraded":"2024-09-10T20:59:48.009545+0000","last_fullsized":"2024-09-10T20:59:48.009545+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:09.439607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371718+0000","last_change":"2024-09-10T20:59:46.012513+0000","last_active":"2024-09-10T20:59:48.371718+0000","last_peered":"2024-09-10T20:59:48.371718+0000","last_clean":"2024-09-10T20:59:48.371718+0000","last_became_active":"2024-09-10T20:59:46.012251+0000","last_became_peered":"2024-09-10T20:59:46.012251+0000","last_unstale":"2024-09-10T20:59:48.371718+0000","last_undegraded":"2024-09-10T20:59:48.371718+0000","last_fullsized":"2024-09-10T20:59:48.371718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:06:09.764168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008181+0000","last_change":"2024-09-10T20:59:47.027750+0000","last_active":"2024-09-10T20:59:48.008181+0000","last_peered":"2024-09-10T20:59:48.008181+0000","last_clean":"2024-09-10T20:59:48.008181+0000","last_became_active":"2024-09-10T20:59:47.027454+0000","last_became_peered":"2024-09-10T20:59:47.027454+0000","last_unstale":"2024-09-10T20:59:48.008181+0000","last_undegraded":"2024-09-10T20:59:48.008181+0000","last_fullsized":"2024-09-10T20:59:48.008181+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:48.206623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008727+0000","last_change":"2024-09-10T20:59:46.009919+0000","last_active":"2024-09-10T20:59:48.008727+0000","last_peered":"2024-09-10T20:59:48.008727+0000","last_clean":"2024-09-10T20:59:48.008727+0000","last_became_active":"2024-09-10T20:59:46.009469+0000","last_became_peered":"2024-09-10T20:59:46.009469+0000","last_unstale":"2024-09-10T20:59:48.008727+0000","last_undegraded":"2024-09-10T20:59:48.008727+0000","last_fullsized":"2024-09-10T20:59:48.008727+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:51.458006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024638+0000","last_change":"2024-09-10T20:59:47.030452+0000","last_active":"2024-09-10T20:59:48.024638+0000","last_peered":"2024-09-10T20:59:48.024638+0000","last_clean":"2024-09-10T20:59:48.024638+0000","last_became_active":"2024-09-10T20:59:47.030133+0000","last_became_peered":"2024-09-10T20:59:47.030133+0000","last_unstale":"2024-09-10T20:59:48.024638+0000","last_undegraded":"2024-09-10T20:59:48.024638+0000","last_fullsized":"2024-09-10T20:59:48.024638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:56.952358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008556+0000","last_change":"2024-09-10T20:59:46.014818+0000","last_active":"2024-09-10T20:59:48.008556+0000","last_peered":"2024-09-10T20:59:48.008556+0000","last_clean":"2024-09-10T20:59:48.008556+0000","last_became_active":"2024-09-10T20:59:46.014578+0000","last_became_peered":"2024-09-10T20:59:46.014578+0000","last_unstale":"2024-09-10T20:59:48.008556+0000","last_undegraded":"2024-09-10T20:59:48.008556+0000","last_fullsized":"2024-09-10T20:59:48.008556+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:30.078404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.372663+0000","last_change":"2024-09-10T20:59:47.034303+0000","last_active":"2024-09-10T20:59:48.372663+0000","last_peered":"2024-09-10T20:59:48.372663+0000","last_clean":"2024-09-10T20:59:48.372663+0000","last_became_active":"2024-09-10T20:59:47.033631+0000","last_became_peered":"2024-09-10T20:59:47.033631+0000","last_unstale":"2024-09-10T20:59:48.372663+0000","last_undegraded":"2024-09-10T20:59:48.372663+0000","last_fullsized":"2024-09-10T20:59:48.372663+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:53.031901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010561+0000","last_change":"2024-09-10T20:59:46.007567+0000","last_active":"2024-09-10T20:59:48.010561+0000","last_peered":"2024-09-10T20:59:48.010561+0000","last_clean":"2024-09-10T20:59:48.010561+0000","last_became_active":"2024-09-10T20:59:46.006773+0000","last_became_peered":"2024-09-10T20:59:46.006773+0000","last_unstale":"2024-09-10T20:59:48.010561+0000","last_undegraded":"2024-09-10T20:59:48.010561+0000","last_fullsized":"2024-09-10T20:59:48.010561+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:19.491620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371345+0000","last_change":"2024-09-10T20:59:47.027217+0000","last_active":"2024-09-10T20:59:48.371345+0000","last_peered":"2024-09-10T20:59:48.371345+0000","last_clean":"2024-09-10T20:59:48.371345+0000","last_became_active":"2024-09-10T20:59:47.026939+0000","last_became_peered":"2024-09-10T20:59:47.026939+0000","last_unstale":"2024-09-10T20:59:48.371345+0000","last_undegraded":"2024-09-10T20:59:48.371345+0000","last_fullsized":"2024-09-10T20:59:48.371345+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:25.592848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009689+0000","last_change":"2024-09-10T20:59:46.010441+0000","last_active":"2024-09-10T20:59:48.009689+0000","last_peered":"2024-09-10T20:59:48.009689+0000","last_clean":"2024-09-10T20:59:48.009689+0000","last_became_active":"2024-09-10T20:59:46.010052+0000","last_became_peered":"2024-09-10T20:59:46.010052+0000","last_unstale":"2024-09-10T20:59:48.009689+0000","last_undegraded":"2024-09-10T20:59:48.009689+0000","last_fullsized":"2024-09-10T20:59:48.009689+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:19.191818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010361+0000","last_change":"2024-09-10T20:59:47.037654+0000","last_active":"2024-09-10T20:59:48.010361+0000","last_peered":"2024-09-10T20:59:48.010361+0000","last_clean":"2024-09-10T20:59:48.010361+0000","last_became_active":"2024-09-10T20:59:47.037367+0000","last_became_peered":"2024-09-10T20:59:47.037367+0000","last_unstale":"2024-09-10T20:59:48.010361+0000","last_undegraded":"2024-09-10T20:59:48.010361+0000","last_fullsized":"2024-09-10T20:59:48.010361+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:27.981732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008249+0000","last_change":"2024-09-10T20:59:46.008318+0000","last_active":"2024-09-10T20:59:48.008249+0000","last_peered":"2024-09-10T20:59:48.008249+0000","last_clean":"2024-09-10T20:59:48.008249+0000","last_became_active":"2024-09-10T20:59:46.008069+0000","last_became_peered":"2024-09-10T20:59:46.008069+0000","last_unstale":"2024-09-10T20:59:48.008249+0000","last_undegraded":"2024-09-10T20:59:48.008249+0000","last_fullsized":"2024-09-10T20:59:48.008249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:51.723589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371531+0000","last_change":"2024-09-10T20:59:47.028617+0000","last_active":"2024-09-10T20:59:48.371531+0000","last_peered":"2024-09-10T20:59:48.371531+0000","last_clean":"2024-09-10T20:59:48.371531+0000","last_became_active":"2024-09-10T20:59:47.028282+0000","last_became_peered":"2024-09-10T20:59:47.028282+0000","last_unstale":"2024-09-10T20:59:48.371531+0000","last_undegraded":"2024-09-10T20:59:48.371531+0000","last_fullsized":"2024-09-10T20:59:48.371531+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:35.832692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008386+0000","last_change":"2024-09-10T20:59:46.016007+0000","last_active":"2024-09-10T20:59:48.008386+0000","last_peered":"2024-09-10T20:59:48.008386+0000","last_clean":"2024-09-10T20:59:48.008386+0000","last_became_active":"2024-09-10T20:59:46.015529+0000","last_became_peered":"2024-09-10T20:59:46.015529+0000","last_unstale":"2024-09-10T20:59:48.008386+0000","last_undegraded":"2024-09-10T20:59:48.008386+0000","last_fullsized":"2024-09-10T20:59:48.008386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:58.373425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008159+0000","last_change":"2024-09-10T20:59:47.018746+0000","last_active":"2024-09-10T20:59:48.008159+0000","last_peered":"2024-09-10T20:59:48.008159+0000","last_clean":"2024-09-10T20:59:48.008159+0000","last_became_active":"2024-09-10T20:59:47.018482+0000","last_became_peered":"2024-09-10T20:59:47.018482+0000","last_unstale":"2024-09-10T20:59:48.008159+0000","last_undegraded":"2024-09-10T20:59:48.008159+0000","last_fullsized":"2024-09-10T20:59:48.008159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:54:17.605108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371407+0000","last_change":"2024-09-10T20:59:46.013441+0000","last_active":"2024-09-10T20:59:48.371407+0000","last_peered":"2024-09-10T20:59:48.371407+0000","last_clean":"2024-09-10T20:59:48.371407+0000","last_became_active":"2024-09-10T20:59:46.012202+0000","last_became_peered":"2024-09-10T20:59:46.012202+0000","last_unstale":"2024-09-10T20:59:48.371407+0000","last_undegraded":"2024-09-10T20:59:48.371407+0000","last_fullsized":"2024-09-10T20:59:48.371407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:25:59.817251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371796+0000","last_change":"2024-09-10T20:59:47.025607+0000","last_active":"2024-09-10T20:59:48.371796+0000","last_peered":"2024-09-10T20:59:48.371796+0000","last_clean":"2024-09-10T20:59:48.371796+0000","last_became_active":"2024-09-10T20:59:47.025060+0000","last_became_peered":"2024-09-10T20:59:47.025060+0000","last_unstale":"2024-09-10T20:59:48.371796+0000","last_undegraded":"2024-09-10T20:59:48.371796+0000","last_fullsized":"2024-09-10T20:59:48.371796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:35.047921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371629+0000","last_change":"2024-09-10T20:59:46.008417+0000","last_active":"2024-09-10T20:59:48.371629+0000","last_peered":"2024-09-10T20:59:48.371629+0000","last_clean":"2024-09-10T20:59:48.371629+0000","last_became_active":"2024-09-10T20:59:46.008173+0000","last_became_peered":"2024-09-10T20:59:46.008173+0000","last_unstale":"2024-09-10T20:59:48.371629+0000","last_undegraded":"2024-09-10T20:59:48.371629+0000","last_fullsized":"2024-09-10T20:59:48.371629+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:06.630212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008204+0000","last_change":"2024-09-10T20:59:47.018576+0000","last_active":"2024-09-10T20:59:48.008204+0000","last_peered":"2024-09-10T20:59:48.008204+0000","last_clean":"2024-09-10T20:59:48.008204+0000","last_became_active":"2024-09-10T20:59:47.018285+0000","last_became_peered":"2024-09-10T20:59:47.018285+0000","last_unstale":"2024-09-10T20:59:48.008204+0000","last_undegraded":"2024-09-10T20:59:48.008204+0000","last_fullsized":"2024-09-10T20:59:48.008204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:32.917475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":23,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371682+0000","last_change":"2024-09-10T20:59:46.007998+0000","last_active":"2024-09-10T20:59:48.371682+0000","last_peered":"2024-09-10T20:59:48.371682+0000","last_clean":"2024-09-10T20:59:48.371682+0000","last_became_active":"2024-09-10T20:59:46.007735+0000","last_became_peered":"2024-09-10T20:59:46.007735+0000","last_unstale":"2024-09-10T20:59:48.371682+0000","last_undegraded":"2024-09-10T20:59:48.371682+0000","last_fullsized":"2024-09-10T20:59:48.371682+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:54.320753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008333+0000","last_change":"2024-09-10T20:59:47.037762+0000","last_active":"2024-09-10T20:59:48.008333+0000","last_peered":"2024-09-10T20:59:48.008333+0000","last_clean":"2024-09-10T20:59:48.008333+0000","last_became_active":"2024-09-10T20:59:47.037007+0000","last_became_peered":"2024-09-10T20:59:47.037007+0000","last_unstale":"2024-09-10T20:59:48.008333+0000","last_undegraded":"2024-09-10T20:59:48.008333+0000","last_fullsized":"2024-09-10T20:59:48.008333+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:03.943638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010092+0000","last_change":"2024-09-10T20:59:46.009743+0000","last_active":"2024-09-10T20:59:48.010092+0000","last_peered":"2024-09-10T20:59:48.010092+0000","last_clean":"2024-09-10T20:59:48.010092+0000","last_became_active":"2024-09-10T20:59:46.009491+0000","last_became_peered":"2024-09-10T20:59:46.009491+0000","last_unstale":"2024-09-10T20:59:48.010092+0000","last_undegraded":"2024-09-10T20:59:48.010092+0000","last_fullsized":"2024-09-10T20:59:48.010092+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:56.031945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009586+0000","last_change":"2024-09-10T20:59:47.026805+0000","last_active":"2024-09-10T20:59:48.009586+0000","last_peered":"2024-09-10T20:59:48.009586+0000","last_clean":"2024-09-10T20:59:48.009586+0000","last_became_active":"2024-09-10T20:59:47.026545+0000","last_became_peered":"2024-09-10T20:59:47.026545+0000","last_unstale":"2024-09-10T20:59:48.009586+0000","last_undegraded":"2024-09-10T20:59:48.009586+0000","last_fullsized":"2024-09-10T20:59:48.009586+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:47.670364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010014+0000","last_change":"2024-09-10T20:59:46.012384+0000","last_active":"2024-09-10T20:59:48.010014+0000","last_peered":"2024-09-10T20:59:48.010014+0000","last_clean":"2024-09-10T20:59:48.010014+0000","last_became_active":"2024-09-10T20:59:46.012119+0000","last_became_peered":"2024-09-10T20:59:46.012119+0000","last_unstale":"2024-09-10T20:59:48.010014+0000","last_undegraded":"2024-09-10T20:59:48.010014+0000","last_fullsized":"2024-09-10T20:59:48.010014+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:55.392323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009751+0000","last_change":"2024-09-10T20:59:47.033830+0000","last_active":"2024-09-10T20:59:48.009751+0000","last_peered":"2024-09-10T20:59:48.009751+0000","last_clean":"2024-09-10T20:59:48.009751+0000","last_became_active":"2024-09-10T20:59:47.033539+0000","last_became_peered":"2024-09-10T20:59:47.033539+0000","last_unstale":"2024-09-10T20:59:48.009751+0000","last_undegraded":"2024-09-10T20:59:48.009751+0000","last_fullsized":"2024-09-10T20:59:48.009751+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:25.592848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024663+0000","last_change":"2024-09-10T20:59:46.010713+0000","last_active":"2024-09-10T20:59:48.024663+0000","last_peered":"2024-09-10T20:59:48.024663+0000","last_clean":"2024-09-10T20:59:48.024663+0000","last_became_active":"2024-09-10T20:59:46.010448+0000","last_became_peered":"2024-09-10T20:59:46.010448+0000","last_unstale":"2024-09-10T20:59:48.024663+0000","last_undegraded":"2024-09-10T20:59:48.024663+0000","last_fullsized":"2024-09-10T20:59:48.024663+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:41:37.633483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024766+0000","last_change":"2024-09-10T20:59:47.018725+0000","last_active":"2024-09-10T20:59:48.024766+0000","last_peered":"2024-09-10T20:59:48.024766+0000","last_clean":"2024-09-10T20:59:48.024766+0000","last_became_active":"2024-09-10T20:59:47.018486+0000","last_became_peered":"2024-09-10T20:59:47.018486+0000","last_unstale":"2024-09-10T20:59:48.024766+0000","last_undegraded":"2024-09-10T20:59:48.024766+0000","last_fullsized":"2024-09-10T20:59:48.024766+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:37.582285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371690+0000","last_change":"2024-09-10T20:59:46.013035+0000","last_active":"2024-09-10T20:59:48.371690+0000","last_peered":"2024-09-10T20:59:48.371690+0000","last_clean":"2024-09-10T20:59:48.371690+0000","last_became_active":"2024-09-10T20:59:46.012519+0000","last_became_peered":"2024-09-10T20:59:46.012519+0000","last_unstale":"2024-09-10T20:59:48.371690+0000","last_undegraded":"2024-09-10T20:59:48.371690+0000","last_fullsized":"2024-09-10T20:59:48.371690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:38.211184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371860+0000","last_change":"2024-09-10T20:59:47.021349+0000","last_active":"2024-09-10T20:59:48.371860+0000","last_peered":"2024-09-10T20:59:48.371860+0000","last_clean":"2024-09-10T20:59:48.371860+0000","last_became_active":"2024-09-10T20:59:47.021068+0000","last_became_peered":"2024-09-10T20:59:47.021068+0000","last_unstale":"2024-09-10T20:59:48.371860+0000","last_undegraded":"2024-09-10T20:59:48.371860+0000","last_fullsized":"2024-09-10T20:59:48.371860+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:41.061675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024461+0000","last_change":"2024-09-10T20:59:46.015090+0000","last_active":"2024-09-10T20:59:48.024461+0000","last_peered":"2024-09-10T20:59:48.024461+0000","last_clean":"2024-09-10T20:59:48.024461+0000","last_became_active":"2024-09-10T20:59:46.014824+0000","last_became_peered":"2024-09-10T20:59:46.014824+0000","last_unstale":"2024-09-10T20:59:48.024461+0000","last_undegraded":"2024-09-10T20:59:48.024461+0000","last_fullsized":"2024-09-10T20:59:48.024461+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:44.918062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371633+0000","last_change":"2024-09-10T20:59:47.028376+0000","last_active":"2024-09-10T20:59:48.371633+0000","last_peered":"2024-09-10T20:59:48.371633+0000","last_clean":"2024-09-10T20:59:48.371633+0000","last_became_active":"2024-09-10T20:59:47.027880+0000","last_became_peered":"2024-09-10T20:59:47.027880+0000","last_unstale":"2024-09-10T20:59:48.371633+0000","last_undegraded":"2024-09-10T20:59:48.371633+0000","last_fullsized":"2024-09-10T20:59:48.371633+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:24.095188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024456+0000","last_change":"2024-09-10T20:59:46.011199+0000","last_active":"2024-09-10T20:59:48.024456+0000","last_peered":"2024-09-10T20:59:48.024456+0000","last_clean":"2024-09-10T20:59:48.024456+0000","last_became_active":"2024-09-10T20:59:46.010933+0000","last_became_peered":"2024-09-10T20:59:46.010933+0000","last_unstale":"2024-09-10T20:59:48.024456+0000","last_undegraded":"2024-09-10T20:59:48.024456+0000","last_fullsized":"2024-09-10T20:59:48.024456+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:37.482570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009795+0000","last_change":"2024-09-10T20:59:47.031076+0000","last_active":"2024-09-10T20:59:48.009795+0000","last_peered":"2024-09-10T20:59:48.009795+0000","last_clean":"2024-09-10T20:59:48.009795+0000","last_became_active":"2024-09-10T20:59:47.030753+0000","last_became_peered":"2024-09-10T20:59:47.030753+0000","last_unstale":"2024-09-10T20:59:48.009795+0000","last_undegraded":"2024-09-10T20:59:48.009795+0000","last_fullsized":"2024-09-10T20:59:48.009795+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:43.434063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.372054+0000","last_change":"2024-09-10T20:59:46.005422+0000","last_active":"2024-09-10T20:59:48.372054+0000","last_peered":"2024-09-10T20:59:48.372054+0000","last_clean":"2024-09-10T20:59:48.372054+0000","last_became_active":"2024-09-10T20:59:46.005163+0000","last_became_peered":"2024-09-10T20:59:46.005163+0000","last_unstale":"2024-09-10T20:59:48.372054+0000","last_undegraded":"2024-09-10T20:59:48.372054+0000","last_fullsized":"2024-09-10T20:59:48.372054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:36.274291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008402+0000","last_change":"2024-09-10T20:59:47.016933+0000","last_active":"2024-09-10T20:59:48.008402+0000","last_peered":"2024-09-10T20:59:48.008402+0000","last_clean":"2024-09-10T20:59:48.008402+0000","last_became_active":"2024-09-10T20:59:47.016668+0000","last_became_peered":"2024-09-10T20:59:47.016668+0000","last_unstale":"2024-09-10T20:59:48.008402+0000","last_undegraded":"2024-09-10T20:59:48.008402+0000","last_fullsized":"2024-09-10T20:59:48.008402+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:02.940429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.024373+0000","last_change":"2024-09-10T20:59:46.015133+0000","last_active":"2024-09-10T20:59:48.024373+0000","last_peered":"2024-09-10T20:59:48.024373+0000","last_clean":"2024-09-10T20:59:48.024373+0000","last_became_active":"2024-09-10T20:59:46.014855+0000","last_became_peered":"2024-09-10T20:59:46.014855+0000","last_unstale":"2024-09-10T20:59:48.024373+0000","last_undegraded":"2024-09-10T20:59:48.024373+0000","last_fullsized":"2024-09-10T20:59:48.024373+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:16.570644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371608+0000","last_change":"2024-09-10T20:59:47.028767+0000","last_active":"2024-09-10T20:59:48.371608+0000","last_peered":"2024-09-10T20:59:48.371608+0000","last_clean":"2024-09-10T20:59:48.371608+0000","last_became_active":"2024-09-10T20:59:47.028529+0000","last_became_peered":"2024-09-10T20:59:47.028529+0000","last_unstale":"2024-09-10T20:59:48.371608+0000","last_undegraded":"2024-09-10T20:59:48.371608+0000","last_fullsized":"2024-09-10T20:59:48.371608+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:19.645203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.010215+0000","last_change":"2024-09-10T20:59:46.006840+0000","last_active":"2024-09-10T20:59:48.010215+0000","last_peered":"2024-09-10T20:59:48.010215+0000","last_clean":"2024-09-10T20:59:48.010215+0000","last_became_active":"2024-09-10T20:59:46.006570+0000","last_became_peered":"2024-09-10T20:59:46.006570+0000","last_unstale":"2024-09-10T20:59:48.010215+0000","last_undegraded":"2024-09-10T20:59:48.010215+0000","last_fullsized":"2024-09-10T20:59:48.010215+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:11.450731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.372779+0000","last_change":"2024-09-10T20:59:47.033554+0000","last_active":"2024-09-10T20:59:48.372779+0000","last_peered":"2024-09-10T20:59:48.372779+0000","last_clean":"2024-09-10T20:59:48.372779+0000","last_became_active":"2024-09-10T20:59:47.033209+0000","last_became_peered":"2024-09-10T20:59:47.033209+0000","last_unstale":"2024-09-10T20:59:48.372779+0000","last_undegraded":"2024-09-10T20:59:48.372779+0000","last_fullsized":"2024-09-10T20:59:48.372779+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:09.469303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.026242+0000","last_change":"2024-09-10T20:59:46.013179+0000","last_active":"2024-09-10T20:59:48.026242+0000","last_peered":"2024-09-10T20:59:48.026242+0000","last_clean":"2024-09-10T20:59:48.026242+0000","last_became_active":"2024-09-10T20:59:46.012907+0000","last_became_peered":"2024-09-10T20:59:46.012907+0000","last_unstale":"2024-09-10T20:59:48.026242+0000","last_undegraded":"2024-09-10T20:59:48.026242+0000","last_fullsized":"2024-09-10T20:59:48.026242+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:19.289722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008396+0000","last_change":"2024-09-10T20:59:47.034104+0000","last_active":"2024-09-10T20:59:48.008396+0000","last_peered":"2024-09-10T20:59:48.008396+0000","last_clean":"2024-09-10T20:59:48.008396+0000","last_became_active":"2024-09-10T20:59:47.033795+0000","last_became_peered":"2024-09-10T20:59:47.033795+0000","last_unstale":"2024-09-10T20:59:48.008396+0000","last_undegraded":"2024-09-10T20:59:48.008396+0000","last_fullsized":"2024-09-10T20:59:48.008396+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:59.999686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371561+0000","last_change":"2024-09-10T20:59:46.017352+0000","last_active":"2024-09-10T20:59:48.371561+0000","last_peered":"2024-09-10T20:59:48.371561+0000","last_clean":"2024-09-10T20:59:48.371561+0000","last_became_active":"2024-09-10T20:59:46.017073+0000","last_became_peered":"2024-09-10T20:59:46.017073+0000","last_unstale":"2024-09-10T20:59:48.371561+0000","last_undegraded":"2024-09-10T20:59:48.371561+0000","last_fullsized":"2024-09-10T20:59:48.371561+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:37.017258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371532+0000","last_change":"2024-09-10T20:59:47.025517+0000","last_active":"2024-09-10T20:59:48.371532+0000","last_peered":"2024-09-10T20:59:48.371532+0000","last_clean":"2024-09-10T20:59:48.371532+0000","last_became_active":"2024-09-10T20:59:47.025246+0000","last_became_peered":"2024-09-10T20:59:47.025246+0000","last_unstale":"2024-09-10T20:59:48.371532+0000","last_undegraded":"2024-09-10T20:59:48.371532+0000","last_fullsized":"2024-09-10T20:59:48.371532+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:32.064402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371860+0000","last_change":"2024-09-10T20:59:46.015703+0000","last_active":"2024-09-10T20:59:48.371860+0000","last_peered":"2024-09-10T20:59:48.371860+0000","last_clean":"2024-09-10T20:59:48.371860+0000","last_became_active":"2024-09-10T20:59:46.015257+0000","last_became_peered":"2024-09-10T20:59:46.015257+0000","last_unstale":"2024-09-10T20:59:48.371860+0000","last_undegraded":"2024-09-10T20:59:48.371860+0000","last_fullsized":"2024-09-10T20:59:48.371860+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:24:41.884818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371619+0000","last_change":"2024-09-10T20:59:47.031083+0000","last_active":"2024-09-10T20:59:48.371619+0000","last_peered":"2024-09-10T20:59:48.371619+0000","last_clean":"2024-09-10T20:59:48.371619+0000","last_became_active":"2024-09-10T20:59:47.030792+0000","last_became_peered":"2024-09-10T20:59:47.030792+0000","last_unstale":"2024-09-10T20:59:48.371619+0000","last_undegraded":"2024-09-10T20:59:48.371619+0000","last_fullsized":"2024-09-10T20:59:48.371619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:56.934193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.372025+0000","last_change":"2024-09-10T20:59:46.015201+0000","last_active":"2024-09-10T20:59:48.372025+0000","last_peered":"2024-09-10T20:59:48.372025+0000","last_clean":"2024-09-10T20:59:48.372025+0000","last_became_active":"2024-09-10T20:59:46.014949+0000","last_became_peered":"2024-09-10T20:59:46.014949+0000","last_unstale":"2024-09-10T20:59:48.372025+0000","last_undegraded":"2024-09-10T20:59:48.372025+0000","last_fullsized":"2024-09-10T20:59:48.372025+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:03.198703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008955+0000","last_change":"2024-09-10T20:59:47.034350+0000","last_active":"2024-09-10T20:59:48.008955+0000","last_peered":"2024-09-10T20:59:48.008955+0000","last_clean":"2024-09-10T20:59:48.008955+0000","last_became_active":"2024-09-10T20:59:47.034093+0000","last_became_peered":"2024-09-10T20:59:47.034093+0000","last_unstale":"2024-09-10T20:59:48.008955+0000","last_undegraded":"2024-09-10T20:59:48.008955+0000","last_fullsized":"2024-09-10T20:59:48.008955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:48.223095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008646+0000","last_change":"2024-09-10T20:59:46.013449+0000","last_active":"2024-09-10T20:59:48.008646+0000","last_peered":"2024-09-10T20:59:48.008646+0000","last_clean":"2024-09-10T20:59:48.008646+0000","last_became_active":"2024-09-10T20:59:46.013161+0000","last_became_peered":"2024-09-10T20:59:46.013161+0000","last_unstale":"2024-09-10T20:59:48.008646+0000","last_undegraded":"2024-09-10T20:59:48.008646+0000","last_fullsized":"2024-09-10T20:59:48.008646+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:06.630212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.372327+0000","last_change":"2024-09-10T20:59:47.034613+0000","last_active":"2024-09-10T20:59:48.372327+0000","last_peered":"2024-09-10T20:59:48.372327+0000","last_clean":"2024-09-10T20:59:48.372327+0000","last_became_active":"2024-09-10T20:59:47.034325+0000","last_became_peered":"2024-09-10T20:59:47.034325+0000","last_unstale":"2024-09-10T20:59:48.372327+0000","last_undegraded":"2024-09-10T20:59:48.372327+0000","last_fullsized":"2024-09-10T20:59:48.372327+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:47.620382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.008062+0000","last_change":"2024-09-10T20:59:46.012810+0000","last_active":"2024-09-10T20:59:48.008062+0000","last_peered":"2024-09-10T20:59:48.008062+0000","last_clean":"2024-09-10T20:59:48.008062+0000","last_became_active":"2024-09-10T20:59:46.012532+0000","last_became_peered":"2024-09-10T20:59:46.012532+0000","last_unstale":"2024-09-10T20:59:48.008062+0000","last_undegraded":"2024-09-10T20:59:48.008062+0000","last_fullsized":"2024-09-10T20:59:48.008062+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:25.702550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.009911+0000","last_change":"2024-09-10T20:59:47.032222+0000","last_active":"2024-09-10T20:59:48.009911+0000","last_peered":"2024-09-10T20:59:48.009911+0000","last_clean":"2024-09-10T20:59:48.009911+0000","last_became_active":"2024-09-10T20:59:47.031463+0000","last_became_peered":"2024-09-10T20:59:47.031463+0000","last_unstale":"2024-09-10T20:59:48.009911+0000","last_undegraded":"2024-09-10T20:59:48.009911+0000","last_fullsized":"2024-09-10T20:59:48.009911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:45.991434+0000","last_clean_scrub_stamp":"2024-09-10T20:59:45.991434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:24.386660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.372500+0000","last_change":"2024-09-10T20:59:46.009765+0000","last_active":"2024-09-10T20:59:48.372500+0000","last_peered":"2024-09-10T20:59:48.372500+0000","last_clean":"2024-09-10T20:59:48.372500+0000","last_became_active":"2024-09-10T20:59:46.009508+0000","last_became_peered":"2024-09-10T20:59:46.009508+0000","last_unstale":"2024-09-10T20:59:48.372500+0000","last_undegraded":"2024-09-10T20:59:48.372500+0000","last_fullsized":"2024-09-10T20:59:48.372500+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:59:44.986452+0000","last_clean_scrub_stamp":"2024-09-10T20:59:44.986452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:04.264652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":146,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T20:59:48.371804+0000","last_change":"2024-09-10T20:58:36.895993+0000","last_active":"2024-09-10T20:59:48.371804+0000","last_peered":"2024-09-10T20:59:48.371804+0000","last_clean":"2024-09-10T20:59:48.371804+0000","last_became_active":"2024-09-10T20:58:36.894988+0000","last_became_peered":"2024-09-10T20:58:36.894988+0000","last_unstale":"2024-09-10T20:59:48.371804+0000","last_undegraded":"2024-09-10T20:59:48.371804+0000","last_fullsized":"2024-09-10T20:59:48.371804+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_clean_scrub_stamp":"2024-09-10T20:58:35.588702+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.082528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1064960,"data_stored":790102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":458752,"data_stored":198634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":462848,"data_stored":198656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":471040,"data_stored":199698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":198122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":198142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1052672,"data_stored":788454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":198164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T20:59:55.025 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T20:59:55.025 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T20:59:55.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.167+0000 7f3a0f09b640 1 -- 172.21.15.98:0/1298375563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a080d2a20 msgr2=0x7f3a080cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.167+0000 7f3a0f09b640 1 --2- 172.21.15.98:0/1298375563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a080d2a20 0x7f3a080cea40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3a00005fd0 tx=0x7f3a0002aa90 comp rx=0 tx=0).stop 2024-09-10T20:59:55.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.167+0000 7f3a0f09b640 1 -- 172.21.15.98:0/1298375563 shutdown_connections 2024-09-10T20:59:55.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.167+0000 7f3a0f09b640 1 --2- 172.21.15.98:0/1298375563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a080d4af0 0x7f3a080d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.167+0000 7f3a0f09b640 1 --2- 172.21.15.98:0/1298375563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a080d2a20 0x7f3a080cea40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.167+0000 7f3a0f09b640 1 -- 172.21.15.98:0/1298375563 >> 172.21.15.98:0/1298375563 conn(0x7f3a080d3f70 msgr2=0x7f3a080d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:55.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.167+0000 7f3a0f09b640 1 -- 172.21.15.98:0/1298375563 shutdown_connections 2024-09-10T20:59:55.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.167+0000 7f3a0f09b640 1 -- 172.21.15.98:0/1298375563 wait complete. 2024-09-10T20:59:55.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.168+0000 7f3a0f09b640 1 Processor -- start 2024-09-10T20:59:55.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.168+0000 7f3a0f09b640 1 -- start start 2024-09-10T20:59:55.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0f09b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a080d2a20 0x7f3a08160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:55.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0f09b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a080d4af0 0x7f3a080cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:55.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0f09b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a0815e060 0x7f3a080cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:55.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0f09b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a080cf100 con 0x7f3a0815e060 2024-09-10T20:59:55.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0f09b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3a080cef80 con 0x7f3a080d2a20 2024-09-10T20:59:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0f09b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3a080cf280 con 0x7f3a080d4af0 2024-09-10T20:59:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0d638640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a0815e060 0x7f3a080cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0d638640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a0815e060 0x7f3a080cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54896/0 (socket says 172.21.15.98:54896) 2024-09-10T20:59:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0d638640 1 -- 172.21.15.98:0/213807810 learned_addr learned my addr 172.21.15.98:0/213807810 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0ce37640 1 --2- 172.21.15.98:0/213807810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a080d2a20 0x7f3a08160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f39fffff640 1 --2- 172.21.15.98:0/213807810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a080d4af0 0x7f3a080cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0d638640 1 -- 172.21.15.98:0/213807810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a080d4af0 msgr2=0x7f3a080cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0d638640 1 --2- 172.21.15.98:0/213807810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a080d4af0 0x7f3a080cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0d638640 1 -- 172.21.15.98:0/213807810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a080d2a20 msgr2=0x7f3a08160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0d638640 1 --2- 172.21.15.98:0/213807810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a080d2a20 0x7f3a08160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.169+0000 7f3a0d638640 1 -- 172.21.15.98:0/213807810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a080cd3d0 con 0x7f3a0815e060 2024-09-10T20:59:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.170+0000 7f3a0d638640 1 --2- 172.21.15.98:0/213807810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a0815e060 0x7f3a080cb580 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3a00006010 tx=0x7f3a00003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.170+0000 7f39fd7fa640 1 -- 172.21.15.98:0/213807810 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00046070 con 0x7f3a0815e060 2024-09-10T20:59:55.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.170+0000 7f39fd7fa640 1 -- 172.21.15.98:0/213807810 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a00003e50 con 0x7f3a0815e060 2024-09-10T20:59:55.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.170+0000 7f3a0f09b640 1 -- 172.21.15.98:0/213807810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a080cd6b0 con 0x7f3a0815e060 2024-09-10T20:59:55.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.170+0000 7f39fd7fa640 1 -- 172.21.15.98:0/213807810 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a000044b0 con 0x7f3a0815e060 2024-09-10T20:59:55.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.170+0000 7f3a0f09b640 1 -- 172.21.15.98:0/213807810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a08161180 con 0x7f3a0815e060 2024-09-10T20:59:55.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.171+0000 7f3a0f09b640 1 -- 172.21.15.98:0/213807810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a080d2840 con 0x7f3a0815e060 2024-09-10T20:59:55.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.171+0000 7f39fd7fa640 1 -- 172.21.15.98:0/213807810 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3a00004650 con 0x7f3a0815e060 2024-09-10T20:59:55.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.172+0000 7f39fd7fa640 1 --2- 172.21.15.98:0/213807810 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f39f4077fa0 0x7f39f407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:55.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.172+0000 7f39fd7fa640 1 -- 172.21.15.98:0/213807810 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f3a000baf40 con 0x7f3a0815e060 2024-09-10T20:59:55.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.173+0000 7f3a0ce37640 1 --2- 172.21.15.98:0/213807810 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f39f4077fa0 0x7f39f407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:55.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.173+0000 7f3a0ce37640 1 --2- 172.21.15.98:0/213807810 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f39f4077fa0 0x7f39f407a460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a04001720 tx=0x7f3a04009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:55.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.177+0000 7f39fd7fa640 1 -- 172.21.15.98:0/213807810 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3a080d2840 con 0x7f3a0815e060 2024-09-10T20:59:55.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.427+0000 7f3a0f09b640 1 -- 172.21.15.98:0/213807810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3a080cfef0 con 0x7f3a0815e060 2024-09-10T20:59:55.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.429+0000 7f39fd7fa640 1 -- 172.21.15.98:0/213807810 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f3a00088e20 con 0x7f3a0815e060 2024-09-10T20:59:55.433 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-10T20:59:55.433 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDKsuBm9hT3OBAAecwpOYTkz+JIO0Xg/GolGw== 2024-09-10T20:59:55.433 INFO:teuthology.orchestra.run.smithi098.stdout: caps mds = "allow" 2024-09-10T20:59:55.433 INFO:teuthology.orchestra.run.smithi098.stdout: caps mon = "allow r" 2024-09-10T20:59:55.433 INFO:teuthology.orchestra.run.smithi098.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T20:59:55.434 INFO:teuthology.orchestra.run.smithi098.stderr:updated caps for client.0 2024-09-10T20:59:55.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.431+0000 7f3a0f09b640 1 -- 172.21.15.98:0/213807810 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f39f4077fa0 msgr2=0x7f39f407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.431+0000 7f3a0f09b640 1 --2- 172.21.15.98:0/213807810 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f39f4077fa0 0x7f39f407a460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3a04001720 tx=0x7f3a04009040 comp rx=0 tx=0).stop 2024-09-10T20:59:55.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.431+0000 7f3a0f09b640 1 -- 172.21.15.98:0/213807810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a0815e060 msgr2=0x7f3a080cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.431+0000 7f3a0f09b640 1 --2- 172.21.15.98:0/213807810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a0815e060 0x7f3a080cb580 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3a00006010 tx=0x7f3a00003ce0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.431+0000 7f3a0f09b640 1 -- 172.21.15.98:0/213807810 shutdown_connections 2024-09-10T20:59:55.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.431+0000 7f3a0f09b640 1 --2- 172.21.15.98:0/213807810 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f39f4077fa0 0x7f39f407a460 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.431+0000 7f3a0f09b640 1 --2- 172.21.15.98:0/213807810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a0815e060 0x7f3a080cb580 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.431+0000 7f3a0f09b640 1 --2- 172.21.15.98:0/213807810 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3a080d4af0 0x7f3a080cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.431+0000 7f3a0f09b640 1 --2- 172.21.15.98:0/213807810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3a080d2a20 0x7f3a08160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.431+0000 7f3a0f09b640 1 -- 172.21.15.98:0/213807810 >> 172.21.15.98:0/213807810 conn(0x7f3a080d3f70 msgr2=0x7f3a080b7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:55.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.431+0000 7f3a0f09b640 1 -- 172.21.15.98:0/213807810 shutdown_connections 2024-09-10T20:59:55.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.431+0000 7f3a0f09b640 1 -- 172.21.15.98:0/213807810 wait complete. 2024-09-10T20:59:55.446 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:59:55.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.545+0000 7f733f165640 1 -- 172.21.15.98:0/2246893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73380d4af0 msgr2=0x7f73380d2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.545+0000 7f733f165640 1 --2- 172.21.15.98:0/2246893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73380d4af0 0x7f73380d2a20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7338069b20 tx=0x7f733002a9c0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.545+0000 7f733f165640 1 -- 172.21.15.98:0/2246893 shutdown_connections 2024-09-10T20:59:55.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.545+0000 7f733f165640 1 --2- 172.21.15.98:0/2246893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73380d4af0 0x7f73380d2a20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.545+0000 7f733f165640 1 --2- 172.21.15.98:0/2246893 >> v2:172.21.15.190:3300/0 conn(0x7f73380b7300 0x7f73380d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.545+0000 7f733f165640 1 --2- 172.21.15.98:0/2246893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f733815e060 0x7f73380b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.545+0000 7f733f165640 1 -- 172.21.15.98:0/2246893 >> 172.21.15.98:0/2246893 conn(0x7f73380d3f70 msgr2=0x7f73380d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:55.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.545+0000 7f733f165640 1 -- 172.21.15.98:0/2246893 shutdown_connections 2024-09-10T20:59:55.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.545+0000 7f733f165640 1 -- 172.21.15.98:0/2246893 wait complete. 2024-09-10T20:59:55.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.546+0000 7f733f165640 1 Processor -- start 2024-09-10T20:59:55.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.546+0000 7f733f165640 1 -- start start 2024-09-10T20:59:55.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.547+0000 7f733f165640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73380d4af0 0x7f73380cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:55.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.547+0000 7f733f165640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733815e060 0x7f733815fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:55.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.547+0000 7f733f165640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73380cc660 0x7f733806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:55.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.547+0000 7f733f165640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7338161e60 con 0x7f733815e060 2024-09-10T20:59:55.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.547+0000 7f733f165640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7338161ce0 con 0x7f73380d4af0 2024-09-10T20:59:55.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.547+0000 7f733f165640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7338161fe0 con 0x7f73380cc660 2024-09-10T20:59:55.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.547+0000 7f7337fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733815e060 0x7f733815fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:55.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.547+0000 7f7337fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733815e060 0x7f733815fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54916/0 (socket says 172.21.15.98:54916) 2024-09-10T20:59:55.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.547+0000 7f7337fff640 1 -- 172.21.15.98:0/1665525688 learned_addr learned my addr 172.21.15.98:0/1665525688 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.547+0000 7f733d702640 1 --2- 172.21.15.98:0/1665525688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73380cc660 0x7f733806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.547+0000 7f733cf01640 1 --2- 172.21.15.98:0/1665525688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73380d4af0 0x7f73380cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.548+0000 7f7337fff640 1 -- 172.21.15.98:0/1665525688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73380cc660 msgr2=0x7f733806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.548+0000 7f7337fff640 1 --2- 172.21.15.98:0/1665525688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73380cc660 0x7f733806d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.548+0000 7f7337fff640 1 -- 172.21.15.98:0/1665525688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73380d4af0 msgr2=0x7f73380cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.548+0000 7f7337fff640 1 --2- 172.21.15.98:0/1665525688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73380d4af0 0x7f73380cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.548+0000 7f7337fff640 1 -- 172.21.15.98:0/1665525688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73380cbf90 con 0x7f733815e060 2024-09-10T20:59:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.548+0000 7f7337fff640 1 --2- 172.21.15.98:0/1665525688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733815e060 0x7f733815fc20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f732400d770 tx=0x7f732400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.548+0000 7f73357fa640 1 -- 172.21.15.98:0/1665525688 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324017070 con 0x7f733815e060 2024-09-10T20:59:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.548+0000 7f73357fa640 1 -- 172.21.15.98:0/1665525688 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7324009d20 con 0x7f733815e060 2024-09-10T20:59:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.548+0000 7f733f165640 1 -- 172.21.15.98:0/1665525688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7338160160 con 0x7f733815e060 2024-09-10T20:59:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.548+0000 7f73357fa640 1 -- 172.21.15.98:0/1665525688 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73240106c0 con 0x7f733815e060 2024-09-10T20:59:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.548+0000 7f733f165640 1 -- 172.21.15.98:0/1665525688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73380ccdd0 con 0x7f733815e060 2024-09-10T20:59:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.549+0000 7f733f165640 1 -- 172.21.15.98:0/1665525688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73380b6dc0 con 0x7f733815e060 2024-09-10T20:59:55.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.550+0000 7f73357fa640 1 -- 172.21.15.98:0/1665525688 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7324010860 con 0x7f733815e060 2024-09-10T20:59:55.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.550+0000 7f73357fa640 1 --2- 172.21.15.98:0/1665525688 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f731c077f70 0x7f731c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:55.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.550+0000 7f73357fa640 1 -- 172.21.15.98:0/1665525688 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f732409a490 con 0x7f733815e060 2024-09-10T20:59:55.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.551+0000 7f733cf01640 1 --2- 172.21.15.98:0/1665525688 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f731c077f70 0x7f731c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:55.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.551+0000 7f733cf01640 1 --2- 172.21.15.98:0/1665525688 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f731c077f70 0x7f731c07a430 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f732c0045b0 tx=0x7f732c00b2e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:55.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.554+0000 7f73357fa640 1 -- 172.21.15.98:0/1665525688 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f73240626a0 con 0x7f733815e060 2024-09-10T20:59:55.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.772+0000 7f733f165640 1 -- 172.21.15.98:0/1665525688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f733808dfa0 con 0x7f733815e060 2024-09-10T20:59:55.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.772+0000 7f73357fa640 1 -- 172.21.15.98:0/1665525688 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7f73240670d0 con 0x7f733815e060 2024-09-10T20:59:55.777 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:55.777 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":17,"btime":"2024-09-10T20:59:49:345393+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/1700293059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":1700293059},{"type":"v1","addr":"172.21.15.98:6834","nonce":1700293059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4048376378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4048376378},{"type":"v1","addr":"172.21.15.190:6835","nonce":4048376378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3087107175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3087107175},{"type":"v1","addr":"172.21.15.190:6837","nonce":3087107175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:59:46.344123+0000","modified":"2024-09-10T20:59:48.682750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5001},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5001":{"gid":5001,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.98:6835/4202850803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":4202850803},{"type":"v1","addr":"172.21.15.98:6835","nonce":4202850803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5001,"qdb_cluster":[5001]},"id":2}]} 2024-09-10T20:59:55.777 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 17 2024-09-10T20:59:55.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.774+0000 7f733f165640 1 -- 172.21.15.98:0/1665525688 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f731c077f70 msgr2=0x7f731c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.774+0000 7f733f165640 1 --2- 172.21.15.98:0/1665525688 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f731c077f70 0x7f731c07a430 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f732c0045b0 tx=0x7f732c00b2e0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.774+0000 7f733f165640 1 -- 172.21.15.98:0/1665525688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733815e060 msgr2=0x7f733815fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.774+0000 7f733f165640 1 --2- 172.21.15.98:0/1665525688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733815e060 0x7f733815fc20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f732400d770 tx=0x7f732400dc30 comp rx=0 tx=0).stop 2024-09-10T20:59:55.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.775+0000 7f733f165640 1 -- 172.21.15.98:0/1665525688 shutdown_connections 2024-09-10T20:59:55.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.775+0000 7f733f165640 1 --2- 172.21.15.98:0/1665525688 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f731c077f70 0x7f731c07a430 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.775+0000 7f733f165640 1 --2- 172.21.15.98:0/1665525688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73380cc660 0x7f733806d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.775+0000 7f733f165640 1 --2- 172.21.15.98:0/1665525688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733815e060 0x7f733815fc20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.775+0000 7f733f165640 1 --2- 172.21.15.98:0/1665525688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73380d4af0 0x7f73380cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.775+0000 7f733f165640 1 -- 172.21.15.98:0/1665525688 >> 172.21.15.98:0/1665525688 conn(0x7f73380d3f70 msgr2=0x7f733806e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:55.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.775+0000 7f733f165640 1 -- 172.21.15.98:0/1665525688 shutdown_connections 2024-09-10T20:59:55.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:55.775+0000 7f733f165640 1 -- 172.21.15.98:0/1665525688 wait complete. 2024-09-10T20:59:56.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T20:59:46.344123+0000', 'modified': '2024-09-10T20:59:48.682750+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5001}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5001': {'gid': 5001, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.98:6835/4202850803', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6832', 'nonce': 4202850803}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 4202850803}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5001, 'qdb_cluster': [5001]} 2024-09-10T20:59:56.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T20:59:56.193 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T20:59:56.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.292+0000 7fdb1e6be640 1 -- 172.21.15.98:0/1675556799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb1806e2e0 msgr2=0x7fdb1815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:56.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.292+0000 7fdb1e6be640 1 --2- 172.21.15.98:0/1675556799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb1806e2e0 0x7fdb1815e530 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c005fd0 tx=0x7fdb0c02a670 comp rx=0 tx=0).stop 2024-09-10T20:59:56.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.292+0000 7fdb1e6be640 1 -- 172.21.15.98:0/1675556799 shutdown_connections 2024-09-10T20:59:56.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.292+0000 7fdb1e6be640 1 --2- 172.21.15.98:0/1675556799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb1806e2e0 0x7fdb1815e530 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.292+0000 7fdb1e6be640 1 -- 172.21.15.98:0/1675556799 >> 172.21.15.98:0/1675556799 conn(0x7fdb180b1760 msgr2=0x7fdb180b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:56.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.292+0000 7fdb1e6be640 1 -- 172.21.15.98:0/1675556799 shutdown_connections 2024-09-10T20:59:56.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.292+0000 7fdb1e6be640 1 -- 172.21.15.98:0/1675556799 wait complete. 2024-09-10T20:59:56.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.293+0000 7fdb1e6be640 1 Processor -- start 2024-09-10T20:59:56.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.293+0000 7fdb1e6be640 1 -- start start 2024-09-10T20:59:56.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.294+0000 7fdb1e6be640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1806e2e0 0x7fdb181f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.294+0000 7fdb1e6be640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb180cf110 0x7fdb18141300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.294+0000 7fdb1e6be640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1815e150 0x7fdb18141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.294+0000 7fdb1e6be640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdb180b4930 con 0x7fdb180cf110 2024-09-10T20:59:56.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.294+0000 7fdb1e6be640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdb180b47b0 con 0x7fdb1815e150 2024-09-10T20:59:56.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.294+0000 7fdb1e6be640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdb180b4ab0 con 0x7fdb1806e2e0 2024-09-10T20:59:56.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.294+0000 7fdb177fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb180cf110 0x7fdb18141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.294+0000 7fdb177fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb180cf110 0x7fdb18141300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54942/0 (socket says 172.21.15.98:54942) 2024-09-10T20:59:56.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.294+0000 7fdb177fe640 1 -- 172.21.15.98:0/2976410235 learned_addr learned my addr 172.21.15.98:0/2976410235 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:56.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.294+0000 7fdb17fff640 1 --2- 172.21.15.98:0/2976410235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1806e2e0 0x7fdb181f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.294+0000 7fdb1cc5b640 1 --2- 172.21.15.98:0/2976410235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1815e150 0x7fdb18141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.294+0000 7fdb177fe640 1 -- 172.21.15.98:0/2976410235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1806e2e0 msgr2=0x7fdb181f2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:56.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.295+0000 7fdb177fe640 1 --2- 172.21.15.98:0/2976410235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1806e2e0 0x7fdb181f2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.295+0000 7fdb177fe640 1 -- 172.21.15.98:0/2976410235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1815e150 msgr2=0x7fdb18141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:56.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.295+0000 7fdb177fe640 1 --2- 172.21.15.98:0/2976410235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1815e150 0x7fdb18141840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.295+0000 7fdb177fe640 1 -- 172.21.15.98:0/2976410235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb181f34a0 con 0x7fdb180cf110 2024-09-10T20:59:56.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.295+0000 7fdb177fe640 1 --2- 172.21.15.98:0/2976410235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb180cf110 0x7fdb18141300 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009850 tx=0x7fdb0c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:56.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.295+0000 7fdb14ff9640 1 -- 172.21.15.98:0/2976410235 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c03f070 con 0x7fdb180cf110 2024-09-10T20:59:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.295+0000 7fdb14ff9640 1 -- 172.21.15.98:0/2976410235 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb0c004890 con 0x7fdb180cf110 2024-09-10T20:59:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.295+0000 7fdb1e6be640 1 -- 172.21.15.98:0/2976410235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb18153cd0 con 0x7fdb180cf110 2024-09-10T20:59:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.295+0000 7fdb14ff9640 1 -- 172.21.15.98:0/2976410235 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c035030 con 0x7fdb180cf110 2024-09-10T20:59:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.295+0000 7fdb1e6be640 1 -- 172.21.15.98:0/2976410235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb18158440 con 0x7fdb180cf110 2024-09-10T20:59:56.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.296+0000 7fdb14ff9640 1 -- 172.21.15.98:0/2976410235 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fdb0c004c60 con 0x7fdb180cf110 2024-09-10T20:59:56.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.297+0000 7fdb14ff9640 1 --2- 172.21.15.98:0/2976410235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdb04077f70 0x7fdb0407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.297+0000 7fdb14ff9640 1 -- 172.21.15.98:0/2976410235 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fdb0c0bb0a0 con 0x7fdb180cf110 2024-09-10T20:59:56.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.297+0000 7fdb17fff640 1 --2- 172.21.15.98:0/2976410235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdb04077f70 0x7fdb0407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.298+0000 7fdb17fff640 1 --2- 172.21.15.98:0/2976410235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdb04077f70 0x7fdb0407a430 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdb08000f60 tx=0x7fdb08009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:56.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.298+0000 7fdb21489740 1 Processor -- start 2024-09-10T20:59:56.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.298+0000 7fdb21489740 1 -- start start 2024-09-10T20:59:56.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb21489740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c948319b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb21489740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 0x55c9483621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb21489740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9483175f0 0x55c948317a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb21489740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c94828e820 con 0x55c9483421b0 2024-09-10T20:59:56.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb21489740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c94827dc50 con 0x55c948361df0 2024-09-10T20:59:56.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb21489740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55c948281a60 con 0x55c9483175f0 2024-09-10T20:59:56.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c948319b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c948319b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54952/0 (socket says 172.21.15.98:54952) 2024-09-10T20:59:56.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb17fff640 1 -- 172.21.15.98:0/962268789 learned_addr learned my addr 172.21.15.98:0/962268789 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:56.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb1cc5b640 1 --2- 172.21.15.98:0/962268789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9483175f0 0x55c948317a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb177fe640 1 --2- 172.21.15.98:0/962268789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 0x55c9483621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb17fff640 1 -- 172.21.15.98:0/962268789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9483175f0 msgr2=0x55c948317a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:56.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb17fff640 1 --2- 172.21.15.98:0/962268789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9483175f0 0x55c948317a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb17fff640 1 -- 172.21.15.98:0/962268789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 msgr2=0x55c9483621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:56.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb17fff640 1 --2- 172.21.15.98:0/962268789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 0x55c9483621f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb17fff640 1 -- 172.21.15.98:0/962268789 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c947fdf850 con 0x55c9483421b0 2024-09-10T20:59:56.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb177fe640 1 --2- 172.21.15.98:0/962268789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 0x55c9483621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:56.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb1cc5b640 1 --2- 172.21.15.98:0/962268789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9483175f0 0x55c948317a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:56.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdb17fff640 1 --2- 172.21.15.98:0/962268789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c948319b20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800f960 tx=0x7fdb08012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687505b113beeb8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:56.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdafdffb640 1 -- 172.21.15.98:0/962268789 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08025070 con 0x55c9483421b0 2024-09-10T20:59:56.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.299+0000 7fdafdffb640 1 -- 172.21.15.98:0/962268789 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb0801b050 con 0x55c9483421b0 2024-09-10T20:59:56.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.300+0000 7fdafdffb640 1 -- 172.21.15.98:0/962268789 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08014e20 con 0x55c9483421b0 2024-09-10T20:59:56.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.300+0000 7fdb21489740 1 -- 172.21.15.98:0/962268789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 msgr2=0x55c948319b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:56.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.300+0000 7fdb21489740 1 --2- 172.21.15.98:0/962268789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c948319b20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800f960 tx=0x7fdb08012000 comp rx=0 tx=0).stop 2024-09-10T20:59:56.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.300+0000 7fdb21489740 1 -- 172.21.15.98:0/962268789 shutdown_connections 2024-09-10T20:59:56.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.300+0000 7fdb21489740 1 --2- 172.21.15.98:0/962268789 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9483175f0 0x55c948317a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.300+0000 7fdb21489740 1 --2- 172.21.15.98:0/962268789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 0x55c9483621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.300+0000 7fdb21489740 1 --2- 172.21.15.98:0/962268789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c948319b20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.300+0000 7fdb21489740 1 -- 172.21.15.98:0/962268789 >> 172.21.15.98:0/962268789 conn(0x55c9483186a0 msgr2=0x55c94831c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:56.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.300+0000 7fdb21489740 1 -- 172.21.15.98:0/962268789 shutdown_connections 2024-09-10T20:59:56.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.300+0000 7fdb21489740 1 -- 172.21.15.98:0/962268789 wait complete. 2024-09-10T20:59:56.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.301+0000 7fdb21489740 1 -- start start 2024-09-10T20:59:56.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.301+0000 7fdb21489740 1 Processor -- start 2024-09-10T20:59:56.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb21489740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9483421b0 0x55c948339c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb21489740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c948361df0 0x55c948373fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb21489740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948374c10 0x55c948374520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb21489740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c947fee620 con 0x55c948361df0 2024-09-10T20:59:56.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb21489740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c9482235f0 con 0x55c948374c10 2024-09-10T20:59:56.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb21489740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55c948316b30 con 0x55c9483421b0 2024-09-10T20:59:56.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb21489740 10 client.0 _ll_register_callbacks cb 0x55c9482270a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T20:59:56.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb177fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c948361df0 0x55c948373fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb177fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c948361df0 0x55c948373fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54966/0 (socket says 172.21.15.98:54966) 2024-09-10T20:59:56.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb177fe640 1 -- 172.21.15.98:0/3931433393 learned_addr learned my addr 172.21.15.98:0/3931433393 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:56.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb17fff640 1 --2- 172.21.15.98:0/3931433393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9483421b0 0x55c948339c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb1cc5b640 1 --2- 172.21.15.98:0/3931433393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948374c10 0x55c948374520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb177fe640 1 -- 172.21.15.98:0/3931433393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9483421b0 msgr2=0x55c948339c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:56.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb177fe640 1 --2- 172.21.15.98:0/3931433393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9483421b0 0x55c948339c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb177fe640 1 -- 172.21.15.98:0/3931433393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948374c10 msgr2=0x55c948374520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:56.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb177fe640 1 --2- 172.21.15.98:0/3931433393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948374c10 0x55c948374520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb177fe640 1 -- 172.21.15.98:0/3931433393 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9480566b0 con 0x55c948361df0 2024-09-10T20:59:56.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb177fe640 1 --2- 172.21.15.98:0/3931433393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c948361df0 0x55c948373fe0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c083350 tx=0x7fdb0c083250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b66d30ee9e659e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:56.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdaf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-10T20:59:56.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdaf6ffd640 1 -- 172.21.15.98:0/3931433393 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c049040 con 0x55c948361df0 2024-09-10T20:59:56.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdaf6ffd640 1 -- 172.21.15.98:0/3931433393 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fdb0c08c380 con 0x55c948361df0 2024-09-10T20:59:56.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.302+0000 7fdb21489740 1 -- 172.21.15.98:0/3931433393 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb0801b050 con 0x55c948361df0 2024-09-10T20:59:56.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.303+0000 7fdaf6ffd640 1 -- 172.21.15.98:0/3931433393 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fdb0c082ed0 con 0x55c948361df0 2024-09-10T20:59:56.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.303+0000 7fdb21489740 10 client.5184 fetch_fsmap learned FSMap version 17 2024-09-10T20:59:56.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.303+0000 7fdb21489740 1 -- 172.21.15.98:0/3931433393 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55c94833fb90 con 0x55c948361df0 2024-09-10T20:59:56.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.303+0000 7fdaf6ffd640 1 -- 172.21.15.98:0/3931433393 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb0801b050 con 0x55c948361df0 2024-09-10T20:59:56.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.303+0000 7fdaf6ffd640 1 -- 172.21.15.98:0/3931433393 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fdb0c082620 con 0x55c948361df0 2024-09-10T20:59:56.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.303+0000 7fdb21489740 10 client.5184 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T20:59:56.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.303+0000 7fdb21489740 10 client.5184 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T20:59:56.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.303+0000 7fdb21489740 20 client.5184 populate_metadata read hostname 'smithi098' 2024-09-10T20:59:56.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.303+0000 7fdb21489740 1 --2- 172.21.15.98:0/3931433393 >> [v2:172.21.15.98:6832/4202850803,v1:172.21.15.98:6835/4202850803] conn(0x55c94837c5c0 0x55c94837e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.303+0000 7fdb21489740 4 client.5184 mds_command: new command op to 5001 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T20:59:56.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.303+0000 7fdb21489740 1 -- 172.21.15.98:0/3931433393 --> [v2:172.21.15.98:6832/4202850803,v1:172.21.15.98:6835/4202850803] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c948290770 con 0x55c94837c5c0 2024-09-10T20:59:56.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.303+0000 7fdb17fff640 1 --2- 172.21.15.98:0/3931433393 >> [v2:172.21.15.98:6832/4202850803,v1:172.21.15.98:6835/4202850803] conn(0x55c94837c5c0 0x55c94837e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.303+0000 7fdaf6ffd640 10 client.5184 ms_handle_connect on v2:172.21.15.98:6832/4202850803 2024-09-10T20:59:56.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.303+0000 7fdb17fff640 1 --2- 172.21.15.98:0/3931433393 >> [v2:172.21.15.98:6832/4202850803,v1:172.21.15.98:6835/4202850803] conn(0x55c94837c5c0 0x55c94837e9b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd5b807b4f35c9bf server_cookie=4303b3e5c421bc2d in_seq=0 out_seq=0 2024-09-10T20:59:56.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdaf6ffd640 1 -- 172.21.15.98:0/3931433393 <== mds.0 v2:172.21.15.98:6832/4202850803 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c948290770 con 0x55c94837c5c0 2024-09-10T20:59:56.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdaf6ffd640 10 client.5184 handle_command_reply: tid=0 2024-09-10T20:59:56.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdb21489740 1 client.5184 shutdown 2024-09-10T20:59:56.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdb21489740 1 -- 172.21.15.98:0/3931433393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c948361df0 msgr2=0x55c948373fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:56.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdb21489740 1 --2- 172.21.15.98:0/3931433393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c948361df0 0x55c948373fe0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c083350 tx=0x7fdb0c083250 comp rx=0 tx=0).stop 2024-09-10T20:59:56.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdb21489740 1 -- 172.21.15.98:0/3931433393 shutdown_connections 2024-09-10T20:59:56.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdb21489740 1 --2- 172.21.15.98:0/3931433393 >> [v2:172.21.15.98:6832/4202850803,v1:172.21.15.98:6835/4202850803] conn(0x55c94837c5c0 0x55c94837e9b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdb21489740 1 --2- 172.21.15.98:0/3931433393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948374c10 0x55c948374520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdb21489740 1 --2- 172.21.15.98:0/3931433393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c948361df0 0x55c948373fe0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdb21489740 1 --2- 172.21.15.98:0/3931433393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c9483421b0 0x55c948339c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdb21489740 1 -- 172.21.15.98:0/3931433393 >> 172.21.15.98:0/3931433393 conn(0x55c9483175f0 msgr2=0x55c94831bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:56.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdaf6ffd640 0 client.5184 ms_handle_reset on v2:172.21.15.98:6832/4202850803 2024-09-10T20:59:56.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdb21489740 1 -- 172.21.15.98:0/3931433393 shutdown_connections 2024-09-10T20:59:56.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdb21489740 1 -- 172.21.15.98:0/3931433393 wait complete. 2024-09-10T20:59:56.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.304+0000 7fdb21489740 20 client.5184 trim_cache size 0 max 16384 2024-09-10T20:59:56.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.322+0000 7fdb21489740 1 Processor -- start 2024-09-10T20:59:56.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.322+0000 7fdb21489740 1 -- start start 2024-09-10T20:59:56.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb21489740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c948363520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb21489740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 0x55c948339eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb21489740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c948374c10 0x55c94831bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb21489740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdb181fe650 con 0x55c9483421b0 2024-09-10T20:59:56.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb21489740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdb18159980 con 0x55c948361df0 2024-09-10T20:59:56.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb21489740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdb181fe000 con 0x55c948374c10 2024-09-10T20:59:56.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c948363520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c948363520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54970/0 (socket says 172.21.15.98:54970) 2024-09-10T20:59:56.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb17fff640 1 -- 172.21.15.98:0/2827893307 learned_addr learned my addr 172.21.15.98:0/2827893307 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:56.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb1cc5b640 1 --2- 172.21.15.98:0/2827893307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c948374c10 0x55c94831bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb177fe640 1 --2- 172.21.15.98:0/2827893307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 0x55c948339eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb17fff640 1 -- 172.21.15.98:0/2827893307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c948374c10 msgr2=0x55c94831bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb17fff640 1 --2- 172.21.15.98:0/2827893307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c948374c10 0x55c94831bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb17fff640 1 -- 172.21.15.98:0/2827893307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 msgr2=0x55c948339eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb17fff640 1 --2- 172.21.15.98:0/2827893307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 0x55c948339eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.323+0000 7fdb17fff640 1 -- 172.21.15.98:0/2827893307 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c948316f10 con 0x55c9483421b0 2024-09-10T20:59:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.324+0000 7fdb17fff640 1 --2- 172.21.15.98:0/2827893307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c948363520 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdb08001e80 tx=0x7fdb08001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c64b0233b3d738a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.324+0000 7fdafd7fa640 1 -- 172.21.15.98:0/2827893307 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08037070 con 0x55c9483421b0 2024-09-10T20:59:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.324+0000 7fdafd7fa640 1 -- 172.21.15.98:0/2827893307 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb0801b050 con 0x55c9483421b0 2024-09-10T20:59:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.324+0000 7fdafd7fa640 1 -- 172.21.15.98:0/2827893307 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0802f070 con 0x55c9483421b0 2024-09-10T20:59:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.324+0000 7fdb21489740 1 -- 172.21.15.98:0/2827893307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 msgr2=0x55c948363520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.324+0000 7fdb21489740 1 --2- 172.21.15.98:0/2827893307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c948363520 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdb08001e80 tx=0x7fdb08001080 comp rx=0 tx=0).stop 2024-09-10T20:59:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.324+0000 7fdb21489740 1 -- 172.21.15.98:0/2827893307 shutdown_connections 2024-09-10T20:59:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.324+0000 7fdb21489740 1 --2- 172.21.15.98:0/2827893307 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c948374c10 0x55c94831bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.324+0000 7fdb21489740 1 --2- 172.21.15.98:0/2827893307 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 0x55c948339eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.324+0000 7fdb21489740 1 --2- 172.21.15.98:0/2827893307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c948363520 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.324+0000 7fdb21489740 1 -- 172.21.15.98:0/2827893307 >> 172.21.15.98:0/2827893307 conn(0x55c9483175f0 msgr2=0x55c948359870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.324+0000 7fdb21489740 1 -- 172.21.15.98:0/2827893307 shutdown_connections 2024-09-10T20:59:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.324+0000 7fdb21489740 1 -- 172.21.15.98:0/2827893307 wait complete. 2024-09-10T20:59:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.325+0000 7fdb21489740 1 -- start start 2024-09-10T20:59:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.325+0000 7fdb21489740 1 Processor -- start 2024-09-10T20:59:56.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb21489740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c9483e43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb21489740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 0x55c948359760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb21489740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c948374c10 0x55c948359ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb21489740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c948358d20 con 0x55c9483421b0 2024-09-10T20:59:56.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb21489740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55c948341130 con 0x55c948361df0 2024-09-10T20:59:56.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb21489740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55c9483407a0 con 0x55c948374c10 2024-09-10T20:59:56.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb21489740 10 client.0 _ll_register_callbacks cb 0x7fdb08007c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T20:59:56.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c9483e43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c9483e43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54972/0 (socket says 172.21.15.98:54972) 2024-09-10T20:59:56.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb17fff640 1 -- 172.21.15.98:0/2795270269 learned_addr learned my addr 172.21.15.98:0/2795270269 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:56.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb177fe640 1 --2- 172.21.15.98:0/2795270269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 0x55c948359760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb1cc5b640 1 --2- 172.21.15.98:0/2795270269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c948374c10 0x55c948359ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb17fff640 1 -- 172.21.15.98:0/2795270269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c948374c10 msgr2=0x55c948359ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:56.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb17fff640 1 --2- 172.21.15.98:0/2795270269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c948374c10 0x55c948359ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb17fff640 1 -- 172.21.15.98:0/2795270269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 msgr2=0x55c948359760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T20:59:56.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb17fff640 1 --2- 172.21.15.98:0/2795270269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 0x55c948359760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb17fff640 1 -- 172.21.15.98:0/2795270269 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c948317e40 con 0x55c9483421b0 2024-09-10T20:59:56.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdb17fff640 1 --2- 172.21.15.98:0/2795270269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c9483e43c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55c948362c30 tx=0x7fdb08014be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71927c674a02a929 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:56.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.326+0000 7fdaf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-10T20:59:56.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdaf6ffd640 1 -- 172.21.15.98:0/2795270269 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08037070 con 0x55c9483421b0 2024-09-10T20:59:56.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdaf6ffd640 1 -- 172.21.15.98:0/2795270269 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fdb0802b710 con 0x55c9483421b0 2024-09-10T20:59:56.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdb21489740 1 -- 172.21.15.98:0/2795270269 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb0801b050 con 0x55c9483421b0 2024-09-10T20:59:56.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdaf6ffd640 1 -- 172.21.15.98:0/2795270269 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fdb08042050 con 0x55c9483421b0 2024-09-10T20:59:56.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdb21489740 10 client.5190 fetch_fsmap learned FSMap version 17 2024-09-10T20:59:56.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdb21489740 1 -- 172.21.15.98:0/2795270269 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55c948362950 con 0x55c9483421b0 2024-09-10T20:59:56.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdaf6ffd640 1 -- 172.21.15.98:0/2795270269 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb0801b050 con 0x55c9483421b0 2024-09-10T20:59:56.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdaf6ffd640 1 -- 172.21.15.98:0/2795270269 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fdb08045020 con 0x55c9483421b0 2024-09-10T20:59:56.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdb21489740 10 client.5190 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T20:59:56.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdb21489740 10 client.5190 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T20:59:56.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdb21489740 20 client.5190 populate_metadata read hostname 'smithi098' 2024-09-10T20:59:56.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdb21489740 1 --2- 172.21.15.98:0/2795270269 >> [v2:172.21.15.98:6832/4202850803,v1:172.21.15.98:6835/4202850803] conn(0x55c9483e5160 0x55c9483e5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdb21489740 4 client.5190 mds_command: new command op to 5001 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T20:59:56.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdb21489740 1 -- 172.21.15.98:0/2795270269 --> [v2:172.21.15.98:6832/4202850803,v1:172.21.15.98:6835/4202850803] -- command(tid 0: {"prefix": "status"}) -- 0x55c94828c220 con 0x55c9483e5160 2024-09-10T20:59:56.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdb177fe640 1 --2- 172.21.15.98:0/2795270269 >> [v2:172.21.15.98:6832/4202850803,v1:172.21.15.98:6835/4202850803] conn(0x55c9483e5160 0x55c9483e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdaf6ffd640 10 client.5190 ms_handle_connect on v2:172.21.15.98:6832/4202850803 2024-09-10T20:59:56.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.327+0000 7fdb177fe640 1 --2- 172.21.15.98:0/2795270269 >> [v2:172.21.15.98:6832/4202850803,v1:172.21.15.98:6835/4202850803] conn(0x55c9483e5160 0x55c9483e5540 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f536fd1a056d452e server_cookie=682279f03e5ec26f in_seq=0 out_seq=0 2024-09-10T20:59:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdaf6ffd640 1 -- 172.21.15.98:0/2795270269 <== mds.0 v2:172.21.15.98:6832/4202850803 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55c94828c220 con 0x55c9483e5160 2024-09-10T20:59:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdaf6ffd640 10 client.5190 handle_command_reply: tid=0 2024-09-10T20:59:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb21489740 1 client.5190 shutdown 2024-09-10T20:59:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb21489740 1 -- 172.21.15.98:0/2795270269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 msgr2=0x55c9483e43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb21489740 1 --2- 172.21.15.98:0/2795270269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c9483e43c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55c948362c30 tx=0x7fdb08014be0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb21489740 1 -- 172.21.15.98:0/2795270269 shutdown_connections 2024-09-10T20:59:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb21489740 1 --2- 172.21.15.98:0/2795270269 >> [v2:172.21.15.98:6832/4202850803,v1:172.21.15.98:6835/4202850803] conn(0x55c9483e5160 0x55c9483e5540 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb21489740 1 --2- 172.21.15.98:0/2795270269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c948374c10 0x55c948359ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdaf6ffd640 0 client.5190 ms_handle_reset on v2:172.21.15.98:6832/4202850803 2024-09-10T20:59:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb21489740 1 --2- 172.21.15.98:0/2795270269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c948361df0 0x55c948359760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb21489740 1 --2- 172.21.15.98:0/2795270269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c9483421b0 0x55c9483e43c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb21489740 1 -- 172.21.15.98:0/2795270269 >> 172.21.15.98:0/2795270269 conn(0x55c9483175f0 msgr2=0x55c948341510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb21489740 1 -- 172.21.15.98:0/2795270269 shutdown_connections 2024-09-10T20:59:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb21489740 1 -- 172.21.15.98:0/2795270269 wait complete. 2024-09-10T20:59:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb21489740 20 client.5190 trim_cache size 0 max 16384 2024-09-10T20:59:56.339 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-10T20:59:56.339 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "41041377-a016-49fd-be1f-323d0b834557", 2024-09-10T20:59:56.339 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-10T20:59:56.339 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 5001, 2024-09-10T20:59:56.339 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-10T20:59:56.339 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-10T20:59:56.339 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-10T20:59:56.339 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 9.3100263529999996, 2024-09-10T20:59:56.339 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 17, 2024-09-10T20:59:56.340 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 40, 2024-09-10T20:59:56.340 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 40, 2024-09-10T20:59:56.340 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 22.579165881000002 2024-09-10T20:59:56.340 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-10T20:59:56.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb1e6be640 1 -- 172.21.15.98:0/2976410235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdb04077f70 msgr2=0x7fdb0407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:56.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb1e6be640 1 --2- 172.21.15.98:0/2976410235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdb04077f70 0x7fdb0407a430 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdb08000f60 tx=0x7fdb08009040 comp rx=0 tx=0).stop 2024-09-10T20:59:56.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb1e6be640 1 -- 172.21.15.98:0/2976410235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb180cf110 msgr2=0x7fdb18141300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:56.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb1e6be640 1 --2- 172.21.15.98:0/2976410235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb180cf110 0x7fdb18141300 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009850 tx=0x7fdb0c0039e0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb1e6be640 1 -- 172.21.15.98:0/2976410235 shutdown_connections 2024-09-10T20:59:56.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb1e6be640 1 --2- 172.21.15.98:0/2976410235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fdb04077f70 0x7fdb0407a430 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb1e6be640 1 --2- 172.21.15.98:0/2976410235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1815e150 0x7fdb18141840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb1e6be640 1 --2- 172.21.15.98:0/2976410235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb180cf110 0x7fdb18141300 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb1e6be640 1 --2- 172.21.15.98:0/2976410235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb1806e2e0 0x7fdb181f2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.328+0000 7fdb1e6be640 1 -- 172.21.15.98:0/2976410235 >> 172.21.15.98:0/2976410235 conn(0x7fdb180b1760 msgr2=0x7fdb180cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:56.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.329+0000 7fdb1e6be640 1 -- 172.21.15.98:0/2976410235 shutdown_connections 2024-09-10T20:59:56.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.329+0000 7fdb1e6be640 1 -- 172.21.15.98:0/2976410235 wait complete. 2024-09-10T20:59:56.343 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T20:59:56.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.439+0000 7f64a007f640 1 -- 172.21.15.98:0/2647809205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6498072680 msgr2=0x7f64980ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:56.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.439+0000 7f64a007f640 1 --2- 172.21.15.98:0/2647809205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6498072680 0x7f64980ceb00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f648c005fd0 tx=0x7f648c02a540 comp rx=0 tx=0).stop 2024-09-10T20:59:56.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.439+0000 7f64a007f640 1 -- 172.21.15.98:0/2647809205 shutdown_connections 2024-09-10T20:59:56.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.439+0000 7f64a007f640 1 --2- 172.21.15.98:0/2647809205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6498072ad0 0x7f64980cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.439+0000 7f64a007f640 1 --2- 172.21.15.98:0/2647809205 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6498072680 0x7f64980ceb00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.439+0000 7f64a007f640 1 -- 172.21.15.98:0/2647809205 >> 172.21.15.98:0/2647809205 conn(0x7f64980b1760 msgr2=0x7f64980b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:56.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.439+0000 7f64a007f640 1 -- 172.21.15.98:0/2647809205 shutdown_connections 2024-09-10T20:59:56.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.439+0000 7f64a007f640 1 -- 172.21.15.98:0/2647809205 wait complete. 2024-09-10T20:59:56.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.440+0000 7f64a007f640 1 Processor -- start 2024-09-10T20:59:56.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.440+0000 7f64a007f640 1 -- start start 2024-09-10T20:59:56.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f64a007f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6498072680 0x7f64981f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f64a007f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6498072ad0 0x7f6498143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f64a007f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6498144780 0x7f64980cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f64a007f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f649815c9f0 con 0x7f6498072680 2024-09-10T20:59:56.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f64a007f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f649815c870 con 0x7f6498144780 2024-09-10T20:59:56.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f64a007f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f649815cb70 con 0x7f6498072ad0 2024-09-10T20:59:56.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f649de1b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6498072680 0x7f64981f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f649de1b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6498072680 0x7f64981f8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54990/0 (socket says 172.21.15.98:54990) 2024-09-10T20:59:56.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f649de1b640 1 -- 172.21.15.98:0/1978622408 learned_addr learned my addr 172.21.15.98:0/1978622408 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:56.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f649e61c640 1 --2- 172.21.15.98:0/1978622408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6498144780 0x7f64980cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f649d61a640 1 --2- 172.21.15.98:0/1978622408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6498072ad0 0x7f6498143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f649de1b640 1 -- 172.21.15.98:0/1978622408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6498072ad0 msgr2=0x7f6498143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:56.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f649de1b640 1 --2- 172.21.15.98:0/1978622408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6498072ad0 0x7f6498143320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f649de1b640 1 -- 172.21.15.98:0/1978622408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6498144780 msgr2=0x7f64980cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:56.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f649de1b640 1 --2- 172.21.15.98:0/1978622408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6498144780 0x7f64980cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.441+0000 7f649de1b640 1 -- 172.21.15.98:0/1978622408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64980ce360 con 0x7f6498072680 2024-09-10T20:59:56.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.442+0000 7f649de1b640 1 --2- 172.21.15.98:0/1978622408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6498072680 0x7f64981f8ee0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f648800c760 tx=0x7f648800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:56.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.442+0000 7f64967fc640 1 -- 172.21.15.98:0/1978622408 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488016020 con 0x7f6498072680 2024-09-10T20:59:56.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.442+0000 7f64967fc640 1 -- 172.21.15.98:0/1978622408 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6488003990 con 0x7f6498072680 2024-09-10T20:59:56.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.442+0000 7f64a007f640 1 -- 172.21.15.98:0/1978622408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64980ce640 con 0x7f6498072680 2024-09-10T20:59:56.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.442+0000 7f64967fc640 1 -- 172.21.15.98:0/1978622408 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488006e60 con 0x7f6498072680 2024-09-10T20:59:56.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.442+0000 7f64a007f640 1 -- 172.21.15.98:0/1978622408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64981f89c0 con 0x7f6498072680 2024-09-10T20:59:56.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.443+0000 7f64a007f640 1 -- 172.21.15.98:0/1978622408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f649815d020 con 0x7f6498072680 2024-09-10T20:59:56.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.444+0000 7f64967fc640 1 -- 172.21.15.98:0/1978622408 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f648800a850 con 0x7f6498072680 2024-09-10T20:59:56.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.444+0000 7f64967fc640 1 --2- 172.21.15.98:0/1978622408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6484077fa0 0x7f648407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:56.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.444+0000 7f64967fc640 1 -- 172.21.15.98:0/1978622408 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f64880999c0 con 0x7f6498072680 2024-09-10T20:59:56.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.444+0000 7f649d61a640 1 --2- 172.21.15.98:0/1978622408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6484077fa0 0x7f648407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:56.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.445+0000 7f649d61a640 1 --2- 172.21.15.98:0/1978622408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6484077fa0 0x7f648407a460 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f648c033000 tx=0x7f648c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:56.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.448+0000 7f64967fc640 1 -- 172.21.15.98:0/1978622408 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f648809be20 con 0x7f6498072680 2024-09-10T20:59:56.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.659+0000 7f64a007f640 1 -- 172.21.15.98:0/1978622408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f64980b4d40 con 0x7f6498072680 2024-09-10T20:59:56.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.660+0000 7f64967fc640 1 -- 172.21.15.98:0/1978622408 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f6488020020 con 0x7f6498072680 2024-09-10T20:59:56.664 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T20:59:56.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.662+0000 7f64a007f640 1 -- 172.21.15.98:0/1978622408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6484077fa0 msgr2=0x7f648407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:56.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.662+0000 7f64a007f640 1 --2- 172.21.15.98:0/1978622408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6484077fa0 0x7f648407a460 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f648c033000 tx=0x7f648c004730 comp rx=0 tx=0).stop 2024-09-10T20:59:56.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.662+0000 7f64a007f640 1 -- 172.21.15.98:0/1978622408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6498072680 msgr2=0x7f64981f8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:56.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.662+0000 7f64a007f640 1 --2- 172.21.15.98:0/1978622408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6498072680 0x7f64981f8ee0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f648800c760 tx=0x7f648800cc20 comp rx=0 tx=0).stop 2024-09-10T20:59:56.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.662+0000 7f64a007f640 1 -- 172.21.15.98:0/1978622408 shutdown_connections 2024-09-10T20:59:56.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.662+0000 7f64a007f640 1 --2- 172.21.15.98:0/1978622408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6484077fa0 0x7f648407a460 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.662+0000 7f64a007f640 1 --2- 172.21.15.98:0/1978622408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6498144780 0x7f64980cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.662+0000 7f64a007f640 1 --2- 172.21.15.98:0/1978622408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6498072ad0 0x7f6498143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.662+0000 7f64a007f640 1 --2- 172.21.15.98:0/1978622408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6498072680 0x7f64981f8ee0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:56.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.662+0000 7f64a007f640 1 -- 172.21.15.98:0/1978622408 >> 172.21.15.98:0/1978622408 conn(0x7f64980b1760 msgr2=0x7f64980b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:56.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.662+0000 7f64a007f640 1 -- 172.21.15.98:0/1978622408 shutdown_connections 2024-09-10T20:59:56.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56.662+0000 7f64a007f640 1 -- 172.21.15.98:0/1978622408 wait complete. 2024-09-10T20:59:56.677 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T20:59:56.677 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T20:59:56.677 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-10T20:59:56.677 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T20:59:56.677 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T20:59:56.677 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T20:59:56.677 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T20:59:56.677 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T20:59:56.677 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T20:59:56.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:56.678 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-10T20:59:56.694 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T20:59:56.694 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T20:59:56.694 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T20:59:56.694 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:56.694 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-10T20:59:56.694 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:56.694 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:56.694 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:56.694 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-10T20:59:56.694 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:56.694 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:56.695 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-10T20:59:56.695 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T20:59:56.695 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:56.695 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T20:59:56.695 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:56.695 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T20:59:56.695 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42727sec preferred_lft 42727sec 2024-09-10T20:59:56.695 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link noprefixroute 2024-09-10T20:59:56.695 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:56.695 INFO:teuthology.orchestra.run.smithi098.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T20:59:56.695 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0a:5a:1d:cc:3c:77 brd ff:ff:ff:ff:ff:ff 2024-09-10T20:59:56.695 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T20:59:56.695 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:56.696 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::e069:b2ff:feb8:4916/64 scope link 2024-09-10T20:59:56.696 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:56.696 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T20:59:56.696 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0a:5a:1d:cc:3c:77 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T20:59:56.696 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::85a:1dff:fecc:3c77/64 scope link 2024-09-10T20:59:56.696 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:56.696 INFO:teuthology.orchestra.run.smithi098.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T20:59:56.696 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 4a:c8:86:56:b7:72 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T20:59:56.696 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::48c8:86ff:fe56:b772/64 scope link 2024-09-10T20:59:56.696 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-10T20:59:56.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:56.697 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T20:59:56.757 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-10T20:59:56.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:56.759 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-10T20:59:56.815 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-10T20:59:56.815 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T20:59:56.815 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T20:59:56.815 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T20:59:56.815 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T20:59:56.816 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T20:59:56.816 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-10T20:59:56.816 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T20:59:56.816 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T20:59:56.816 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T20:59:56.816 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-10T20:59:56.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T20:59:56.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:56.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:56.942 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-10T20:59:56.957 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T20:59:56.957 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T20:59:56.957 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T20:59:56.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:56.957 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-10T20:59:57.025 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T20:59:57.025 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T20:59:57.077 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T20:59:57.078 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T20:59:57.078 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T20:59:57.133 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T20:59:57.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:57.134 DEBUG:teuthology.orchestra.run.smithi098:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:57.217 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:57.217 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T20:59:57.225 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T20:59:57.225 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T20:59:57.225 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-09-10T20:59:57.225 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: trying mount with new device syntax: 0@41041377-a016-49fd-be1f-323d0b834557.cephfs=/ 2024-09-10T20:59:57.225 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.98:3300/172.21.15.190:3300/172.21.15.190:3301" will pass to kernel 2024-09-10T20:59:57.225 INFO:teuthology.orchestra.run.smithi098.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T20:59:57.226 INFO:teuthology.orchestra.run.smithi098.stdout: You just mounted a file system that supports labels which does not 2024-09-10T20:59:57.226 INFO:teuthology.orchestra.run.smithi098.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T20:59:57.226 INFO:teuthology.orchestra.run.smithi098.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T20:59:57.226 INFO:teuthology.orchestra.run.smithi098.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T20:59:57.228 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T20:59:57.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:57.229 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T20:59:57.294 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T20:59:57.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T20:59:57.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:57.394 INFO:teuthology.orchestra.run.smithi098.stdout:client5199 2024-09-10T20:59:57.395 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T20:59:57.395 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T20:59:57.410 DEBUG:teuthology.orchestra.run.smithi098:> stdin-killer -- python3 -c ' 2024-09-10T20:59:57.410 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-10T20:59:57.411 DEBUG:teuthology.orchestra.run.smithi098:> os.mkdir("/home/ubuntu/cephtest/mnt.0/testdir") 2024-09-10T20:59:57.411 DEBUG:teuthology.orchestra.run.smithi098:> fd = os.open("/home/ubuntu/cephtest/mnt.0/testdir", os.O_RDONLY) 2024-09-10T20:59:57.411 DEBUG:teuthology.orchestra.run.smithi098:> os.fchmod(fd, 0o777) 2024-09-10T20:59:57.411 DEBUG:teuthology.orchestra.run.smithi098:> os.fsync(fd) 2024-09-10T20:59:57.411 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-10T20:59:57.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:57.530 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:59:57.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.667+0000 7f0307df3640 1 -- 172.21.15.98:0/1893072651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02e8003ad0 msgr2=0x7f02e8003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:57.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.667+0000 7f0307df3640 1 --2- 172.21.15.98:0/1893072651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02e8003ad0 0x7f02e8003ef0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f02f000c640 tx=0x7f02e8005830 comp rx=0 tx=0).stop 2024-09-10T20:59:57.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.667+0000 7f0307df3640 1 -- 172.21.15.98:0/1893072651 shutdown_connections 2024-09-10T20:59:57.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.667+0000 7f0307df3640 1 --2- 172.21.15.98:0/1893072651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02e8003ad0 0x7f02e8003ef0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:57.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.667+0000 7f0307df3640 1 -- 172.21.15.98:0/1893072651 >> 172.21.15.98:0/1893072651 conn(0x7f03000d3f70 msgr2=0x7f03000d4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:57.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.667+0000 7f0307df3640 1 -- 172.21.15.98:0/1893072651 shutdown_connections 2024-09-10T20:59:57.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.667+0000 7f0307df3640 1 -- 172.21.15.98:0/1893072651 wait complete. 2024-09-10T20:59:57.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.668+0000 7f0307df3640 1 Processor -- start 2024-09-10T20:59:57.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.669+0000 7f0307df3640 1 -- start start 2024-09-10T20:59:57.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.669+0000 7f0307df3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02e8003ad0 0x7f03000b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:57.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.669+0000 7f0307df3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0300160ef0 0x7f0300161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:57.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.669+0000 7f0307df3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03001f7e50 0x7f03001f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:57.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.669+0000 7f0307df3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f030006d460 con 0x7f0300160ef0 2024-09-10T20:59:57.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.669+0000 7f0307df3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f03000d2850 con 0x7f02e8003ad0 2024-09-10T20:59:57.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.669+0000 7f0307df3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f030006d5e0 con 0x7f03001f7e50 2024-09-10T20:59:57.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.669+0000 7f030538e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0300160ef0 0x7f0300161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:57.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.669+0000 7f030538e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0300160ef0 0x7f0300161370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55000/0 (socket says 172.21.15.98:55000) 2024-09-10T20:59:57.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.669+0000 7f030538e640 1 -- 172.21.15.98:0/3366883877 learned_addr learned my addr 172.21.15.98:0/3366883877 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:57.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.669+0000 7f0305b8f640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02e8003ad0 0x7f03000b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:57.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.669+0000 7f0306390640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03001f7e50 0x7f03001f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:57.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.670+0000 7f030538e640 1 -- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03001f7e50 msgr2=0x7f03001f82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:57.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.670+0000 7f030538e640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03001f7e50 0x7f03001f82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:57.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.670+0000 7f030538e640 1 -- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02e8003ad0 msgr2=0x7f03000b9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:57.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.670+0000 7f030538e640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02e8003ad0 0x7f03000b9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:57.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.670+0000 7f030538e640 1 -- 172.21.15.98:0/3366883877 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0300161970 con 0x7f0300160ef0 2024-09-10T20:59:57.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.670+0000 7f0306390640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03001f7e50 0x7f03001f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:57.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.670+0000 7f030538e640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0300160ef0 0x7f0300161370 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f02f400e960 tx=0x7f02f400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:57.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.670+0000 7f02fe7fc640 1 -- 172.21.15.98:0/3366883877 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f400cc60 con 0x7f0300160ef0 2024-09-10T20:59:57.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.670+0000 7f02fe7fc640 1 -- 172.21.15.98:0/3366883877 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02f4011470 con 0x7f0300160ef0 2024-09-10T20:59:57.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.670+0000 7f0307df3640 1 -- 172.21.15.98:0/3366883877 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03001f46b0 con 0x7f0300160ef0 2024-09-10T20:59:57.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.670+0000 7f02fe7fc640 1 -- 172.21.15.98:0/3366883877 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f40118c0 con 0x7f0300160ef0 2024-09-10T20:59:57.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.670+0000 7f0307df3640 1 -- 172.21.15.98:0/3366883877 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03001f4f50 con 0x7f0300160ef0 2024-09-10T20:59:57.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.671+0000 7f0307df3640 1 -- 172.21.15.98:0/3366883877 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f030006db40 con 0x7f0300160ef0 2024-09-10T20:59:57.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.672+0000 7f02fe7fc640 1 -- 172.21.15.98:0/3366883877 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f02f400f040 con 0x7f0300160ef0 2024-09-10T20:59:57.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.672+0000 7f02fe7fc640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f02ec077fa0 0x7f02ec07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:57.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.672+0000 7f02fe7fc640 1 -- 172.21.15.98:0/3366883877 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f02f409a710 con 0x7f0300160ef0 2024-09-10T20:59:57.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.672+0000 7f0305b8f640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f02ec077fa0 0x7f02ec07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:57.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.673+0000 7f0305b8f640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f02ec077fa0 0x7f02ec07a460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02f0033000 tx=0x7f03000d4d70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:57.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.676+0000 7f02fe7fc640 1 -- 172.21.15.98:0/3366883877 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f02f4016030 con 0x7f0300160ef0 2024-09-10T20:59:57.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.886+0000 7f0307df3640 1 -- 172.21.15.98:0/3366883877 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03001f4880 con 0x7f0300160ef0 2024-09-10T20:59:57.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.886+0000 7f02fe7fc640 1 -- 172.21.15.98:0/3366883877 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7f02f4063080 con 0x7f0300160ef0 2024-09-10T20:59:57.891 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:57.891 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":17,"btime":"2024-09-10T20:59:49:345393+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/1700293059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":1700293059},{"type":"v1","addr":"172.21.15.98:6834","nonce":1700293059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4048376378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4048376378},{"type":"v1","addr":"172.21.15.190:6835","nonce":4048376378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3087107175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3087107175},{"type":"v1","addr":"172.21.15.190:6837","nonce":3087107175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:59:46.344123+0000","modified":"2024-09-10T20:59:48.682750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5001},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5001":{"gid":5001,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.98:6835/4202850803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":4202850803},{"type":"v1","addr":"172.21.15.98:6835","nonce":4202850803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5001,"qdb_cluster":[5001]},"id":2}]} 2024-09-10T20:59:57.891 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 17 2024-09-10T20:59:57.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.888+0000 7f0307df3640 1 -- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f02ec077fa0 msgr2=0x7f02ec07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:57.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.888+0000 7f0307df3640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f02ec077fa0 0x7f02ec07a460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02f0033000 tx=0x7f03000d4d70 comp rx=0 tx=0).stop 2024-09-10T20:59:57.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.888+0000 7f0307df3640 1 -- 172.21.15.98:0/3366883877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0300160ef0 msgr2=0x7f0300161370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:57.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.888+0000 7f0307df3640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0300160ef0 0x7f0300161370 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f02f400e960 tx=0x7f02f400ee20 comp rx=0 tx=0).stop 2024-09-10T20:59:57.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.889+0000 7f0307df3640 1 -- 172.21.15.98:0/3366883877 shutdown_connections 2024-09-10T20:59:57.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.889+0000 7f0307df3640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f02ec077fa0 0x7f02ec07a460 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:57.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.889+0000 7f0307df3640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03001f7e50 0x7f03001f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:57.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.889+0000 7f0307df3640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0300160ef0 0x7f0300161370 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:57.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.889+0000 7f0307df3640 1 --2- 172.21.15.98:0/3366883877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f02e8003ad0 0x7f03000b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.889+0000 7f0307df3640 1 -- 172.21.15.98:0/3366883877 >> 172.21.15.98:0/3366883877 conn(0x7f03000d3f70 msgr2=0x7f030015dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.889+0000 7f0307df3640 1 -- 172.21.15.98:0/3366883877 shutdown_connections 2024-09-10T20:59:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:57.889+0000 7f0307df3640 1 -- 172.21.15.98:0/3366883877 wait complete. 2024-09-10T20:59:57.904 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-09-10T20:59:58.023 INFO:teuthology.orchestra.run.smithi098.stdout:{"message":"","return_code":0} 2024-09-10T20:59:58.024 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T20:59:58.024 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T20:59:58.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.121+0000 7f190eb94640 1 -- 172.21.15.98:0/588341846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1908072680 msgr2=0x7f19080ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:58.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.121+0000 7f190eb94640 1 --2- 172.21.15.98:0/588341846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1908072680 0x7f19080ceb00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009830 tx=0x7f18f802a480 comp rx=0 tx=0).stop 2024-09-10T20:59:58.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.121+0000 7f190eb94640 1 -- 172.21.15.98:0/588341846 shutdown_connections 2024-09-10T20:59:58.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.121+0000 7f190eb94640 1 --2- 172.21.15.98:0/588341846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1908072680 0x7f19080ceb00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:58.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.121+0000 7f190eb94640 1 -- 172.21.15.98:0/588341846 >> 172.21.15.98:0/588341846 conn(0x7f19080b1760 msgr2=0x7f19080b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:58.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.121+0000 7f190eb94640 1 -- 172.21.15.98:0/588341846 shutdown_connections 2024-09-10T20:59:58.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.121+0000 7f190eb94640 1 -- 172.21.15.98:0/588341846 wait complete. 2024-09-10T20:59:58.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.122+0000 7f190eb94640 1 Processor -- start 2024-09-10T20:59:58.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.122+0000 7f190eb94640 1 -- start start 2024-09-10T20:59:58.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f190eb94640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1908072680 0x7f19081f8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:58.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f190eb94640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1908072ad0 0x7f190813cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:58.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f190eb94640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1908144780 0x7f1908142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:58.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f190eb94640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f190815cc60 con 0x7f1908072ad0 2024-09-10T20:59:58.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f190eb94640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f19080cf390 con 0x7f1908144780 2024-09-10T20:59:58.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f190eb94640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f190815cde0 con 0x7f1908072680 2024-09-10T20:59:58.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f18fffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1908072ad0 0x7f190813cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:58.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f18fffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1908072ad0 0x7f190813cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:55018/0 (socket says 172.21.15.98:55018) 2024-09-10T20:59:58.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f18fffff640 1 -- 172.21.15.98:0/3283880693 learned_addr learned my addr 172.21.15.98:0/3283880693 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T20:59:58.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f18fffff640 1 -- 172.21.15.98:0/3283880693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1908072680 msgr2=0x7f19081f8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:58.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f18fffff640 1 --2- 172.21.15.98:0/3283880693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1908072680 0x7f19081f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:58.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f18fffff640 1 -- 172.21.15.98:0/3283880693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1908144780 msgr2=0x7f1908142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:58.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f18fffff640 1 --2- 172.21.15.98:0/3283880693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1908144780 0x7f1908142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:58.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f18fffff640 1 -- 172.21.15.98:0/3283880693 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1908143220 con 0x7f1908072ad0 2024-09-10T20:59:58.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f18fffff640 1 --2- 172.21.15.98:0/3283880693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1908072ad0 0x7f190813cb50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009950 tx=0x7f18f8032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:58.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f18fd7fa640 1 -- 172.21.15.98:0/3283880693 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8040070 con 0x7f1908072ad0 2024-09-10T20:59:58.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.123+0000 7f18fd7fa640 1 -- 172.21.15.98:0/3283880693 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18f8003af0 con 0x7f1908072ad0 2024-09-10T20:59:58.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.124+0000 7f190eb94640 1 -- 172.21.15.98:0/3283880693 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19081f4a20 con 0x7f1908072ad0 2024-09-10T20:59:58.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.124+0000 7f18fd7fa640 1 -- 172.21.15.98:0/3283880693 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f8045460 con 0x7f1908072ad0 2024-09-10T20:59:58.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.124+0000 7f190eb94640 1 -- 172.21.15.98:0/3283880693 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19081f4fe0 con 0x7f1908072ad0 2024-09-10T20:59:58.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.124+0000 7f190eb94640 1 -- 172.21.15.98:0/3283880693 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19080cf810 con 0x7f1908072ad0 2024-09-10T20:59:58.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.125+0000 7f18fd7fa640 1 -- 172.21.15.98:0/3283880693 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f18f804d030 con 0x7f1908072ad0 2024-09-10T20:59:58.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.126+0000 7f18fd7fa640 1 --2- 172.21.15.98:0/3283880693 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f18f4077f70 0x7f18f407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:58.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.126+0000 7f18fd7fa640 1 -- 172.21.15.98:0/3283880693 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f18f80bb560 con 0x7f1908072ad0 2024-09-10T20:59:58.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.126+0000 7f190c930640 1 --2- 172.21.15.98:0/3283880693 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f18f4077f70 0x7f18f407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:58.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.126+0000 7f190c930640 1 --2- 172.21.15.98:0/3283880693 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f18f4077f70 0x7f18f407a430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f19080cd2c0 tx=0x7f1904007a30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:58.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.130+0000 7f18fd7fa640 1 -- 172.21.15.98:0/3283880693 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f18f8083f60 con 0x7f1908072ad0 2024-09-10T20:59:58.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.338+0000 7f190eb94640 1 -- 172.21.15.98:0/3283880693 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19080b4d40 con 0x7f1908072ad0 2024-09-10T20:59:58.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.338+0000 7f18fd7fa640 1 -- 172.21.15.98:0/3283880693 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7f18f8083860 con 0x7f1908072ad0 2024-09-10T20:59:58.342 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T20:59:58.343 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":17,"btime":"2024-09-10T20:59:49:345393+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/1700293059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":1700293059},{"type":"v1","addr":"172.21.15.98:6834","nonce":1700293059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4048376378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4048376378},{"type":"v1","addr":"172.21.15.190:6835","nonce":4048376378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4834,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3087107175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3087107175},{"type":"v1","addr":"172.21.15.190:6837","nonce":3087107175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:59:46.344123+0000","modified":"2024-09-10T20:59:48.682750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5001},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5001":{"gid":5001,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.98:6835/4202850803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":4202850803},{"type":"v1","addr":"172.21.15.98:6835","nonce":4202850803}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5001,"qdb_cluster":[5001]},"id":2}]} 2024-09-10T20:59:58.343 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 17 2024-09-10T20:59:58.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.340+0000 7f190eb94640 1 -- 172.21.15.98:0/3283880693 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f18f4077f70 msgr2=0x7f18f407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:58.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.340+0000 7f190eb94640 1 --2- 172.21.15.98:0/3283880693 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f18f4077f70 0x7f18f407a430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f19080cd2c0 tx=0x7f1904007a30 comp rx=0 tx=0).stop 2024-09-10T20:59:58.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.340+0000 7f190eb94640 1 -- 172.21.15.98:0/3283880693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1908072ad0 msgr2=0x7f190813cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:58.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.340+0000 7f190eb94640 1 --2- 172.21.15.98:0/3283880693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1908072ad0 0x7f190813cb50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009950 tx=0x7f18f8032d50 comp rx=0 tx=0).stop 2024-09-10T20:59:58.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.340+0000 7f190eb94640 1 -- 172.21.15.98:0/3283880693 shutdown_connections 2024-09-10T20:59:58.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.340+0000 7f190eb94640 1 --2- 172.21.15.98:0/3283880693 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f18f4077f70 0x7f18f407a430 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:58.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.340+0000 7f190eb94640 1 --2- 172.21.15.98:0/3283880693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1908144780 0x7f1908142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:58.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.340+0000 7f190eb94640 1 --2- 172.21.15.98:0/3283880693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1908072ad0 0x7f190813cb50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:58.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.340+0000 7f190eb94640 1 --2- 172.21.15.98:0/3283880693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1908072680 0x7f19081f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:58.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.340+0000 7f190eb94640 1 -- 172.21.15.98:0/3283880693 >> 172.21.15.98:0/3283880693 conn(0x7f19080b1760 msgr2=0x7f19080b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:58.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.340+0000 7f190eb94640 1 -- 172.21.15.98:0/3283880693 shutdown_connections 2024-09-10T20:59:58.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58.340+0000 7f190eb94640 1 -- 172.21.15.98:0/3283880693 wait complete. 2024-09-10T20:59:58.356 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_log_pause 1 2024-09-10T20:59:58.467 INFO:teuthology.orchestra.run.smithi098.stdout:{"success":"mds_log_pause = '' "} 2024-09-10T20:59:58.467 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_log_pause = '' " } 2024-09-10T20:59:58.468 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-10T20:59:58.468 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-10T20:59:58.468 DEBUG:teuthology.orchestra.run.smithi098:> os.chdir("/home/ubuntu/cephtest/mnt.0/testdir") 2024-09-10T20:59:58.468 DEBUG:teuthology.orchestra.run.smithi098:> os.setgid(65534) 2024-09-10T20:59:58.468 DEBUG:teuthology.orchestra.run.smithi098:> os.setuid(65534) 2024-09-10T20:59:58.468 DEBUG:teuthology.orchestra.run.smithi098:> fd = os.open("testfile", os.O_CREAT | os.O_RDWR, 0o644) 2024-09-10T20:59:58.468 DEBUG:teuthology.orchestra.run.smithi098:> os.fchmod(fd, 0o640) 2024-09-10T20:59:58.468 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-10T20:59:58.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T20:59:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T20:59:58.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T20:59:58.553 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod 600 /home/ubuntu/cephtest/mnt.0/testdir/testfile) 2024-09-10T21:00:08.596 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 2:0 2024-09-10T21:00:08.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.708+0000 7fef2b72b640 1 -- 172.21.15.98:0/4036794013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef100063f0 msgr2=0x7fef10003440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:08.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.708+0000 7fef2b72b640 1 --2- 172.21.15.98:0/4036794013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef100063f0 0x7fef10003440 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fef1c001d90 tx=0x7fef10006940 comp rx=0 tx=0).stop 2024-09-10T21:00:08.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.708+0000 7fef2b72b640 1 -- 172.21.15.98:0/4036794013 shutdown_connections 2024-09-10T21:00:08.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.708+0000 7fef2b72b640 1 --2- 172.21.15.98:0/4036794013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef100063f0 0x7fef10003440 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:08.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.708+0000 7fef2b72b640 1 -- 172.21.15.98:0/4036794013 >> 172.21.15.98:0/4036794013 conn(0x7fef240b1760 msgr2=0x7fef240b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:08.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.708+0000 7fef2b72b640 1 -- 172.21.15.98:0/4036794013 shutdown_connections 2024-09-10T21:00:08.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.708+0000 7fef2b72b640 1 -- 172.21.15.98:0/4036794013 wait complete. 2024-09-10T21:00:08.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.710+0000 7fef2b72b640 1 Processor -- start 2024-09-10T21:00:08.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.710+0000 7fef2b72b640 1 -- start start 2024-09-10T21:00:08.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef2b72b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef100063f0 0x7fef24151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:08.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef2b72b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef2413bed0 0x7fef24154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:08.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef2b72b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef2413c350 0x7fef24154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:08.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef2b72b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fef2406fae0 con 0x7fef2413bed0 2024-09-10T21:00:08.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef2b72b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fef2406f960 con 0x7fef2413c350 2024-09-10T21:00:08.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef2b72b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fef2406fc60 con 0x7fef100063f0 2024-09-10T21:00:08.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef28cc6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef2413bed0 0x7fef24154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:08.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef28cc6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef2413bed0 0x7fef24154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:44186/0 (socket says 172.21.15.98:44186) 2024-09-10T21:00:08.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef28cc6640 1 -- 172.21.15.98:0/2683005175 learned_addr learned my addr 172.21.15.98:0/2683005175 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:08.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef294c7640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef100063f0 0x7fef24151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:08.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef29cc8640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef2413c350 0x7fef24154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:08.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef28cc6640 1 -- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef100063f0 msgr2=0x7fef24151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:08.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef28cc6640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef100063f0 0x7fef24151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:08.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef28cc6640 1 -- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef2413c350 msgr2=0x7fef24154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:08.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef28cc6640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef2413c350 0x7fef24154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:08.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef28cc6640 1 -- 172.21.15.98:0/2683005175 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef24152810 con 0x7fef2413bed0 2024-09-10T21:00:08.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef29cc8640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef2413c350 0x7fef24154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:00:08.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef28cc6640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef2413bed0 0x7fef24154220 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fef14011ab0 tx=0x7fef14011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:08.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef19ffb640 1 -- 172.21.15.98:0/2683005175 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1401e070 con 0x7fef2413bed0 2024-09-10T21:00:08.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef19ffb640 1 -- 172.21.15.98:0/2683005175 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef1400fe70 con 0x7fef2413bed0 2024-09-10T21:00:08.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef2b72b640 1 -- 172.21.15.98:0/2683005175 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef241f50c0 con 0x7fef2413bed0 2024-09-10T21:00:08.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.711+0000 7fef19ffb640 1 -- 172.21.15.98:0/2683005175 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef140042b0 con 0x7fef2413bed0 2024-09-10T21:00:08.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.712+0000 7fef2b72b640 1 -- 172.21.15.98:0/2683005175 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef241f5680 con 0x7fef2413bed0 2024-09-10T21:00:08.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.713+0000 7fef2b72b640 1 -- 172.21.15.98:0/2683005175 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef2415dd20 con 0x7fef2413bed0 2024-09-10T21:00:08.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.713+0000 7fef19ffb640 1 -- 172.21.15.98:0/2683005175 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fef140035f0 con 0x7fef2413bed0 2024-09-10T21:00:08.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.714+0000 7fef19ffb640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fef08077f70 0x7fef0807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:08.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.714+0000 7fef19ffb640 1 -- 172.21.15.98:0/2683005175 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fef1409ab70 con 0x7fef2413bed0 2024-09-10T21:00:08.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.714+0000 7fef294c7640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fef08077f70 0x7fef0807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:08.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.714+0000 7fef294c7640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fef08077f70 0x7fef0807a430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef20001220 tx=0x7fef200067a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:08.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.718+0000 7fef19ffb640 1 -- 172.21.15.98:0/2683005175 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fef14062e90 con 0x7fef2413bed0 2024-09-10T21:00:08.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.962+0000 7fef2b72b640 1 -- 172.21.15.98:0/2683005175 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "2:0"} v 0) -- 0x7fef240b4850 con 0x7fef2413bed0 2024-09-10T21:00:08.966 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-10T21:00:08.962+0000 7fb4184a9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:00:08.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.964+0000 7fef19ffb640 1 -- 172.21.15.98:0/2683005175 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "2:0"}]=0 failed mds gid 5001 v18) ==== 97+0+0 (secure 0 0 0) 0x7fef140678c0 con 0x7fef2413bed0 2024-09-10T21:00:08.968 INFO:teuthology.orchestra.run.smithi098.stderr:failed mds gid 5001 2024-09-10T21:00:08.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.966+0000 7fef2b72b640 1 -- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fef08077f70 msgr2=0x7fef0807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:08.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.966+0000 7fef2b72b640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fef08077f70 0x7fef0807a430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef20001220 tx=0x7fef200067a0 comp rx=0 tx=0).stop 2024-09-10T21:00:08.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.966+0000 7fef2b72b640 1 -- 172.21.15.98:0/2683005175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef2413bed0 msgr2=0x7fef24154220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:08.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.966+0000 7fef2b72b640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef2413bed0 0x7fef24154220 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fef14011ab0 tx=0x7fef14011f70 comp rx=0 tx=0).stop 2024-09-10T21:00:08.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.966+0000 7fef2b72b640 1 -- 172.21.15.98:0/2683005175 shutdown_connections 2024-09-10T21:00:08.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.966+0000 7fef2b72b640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fef08077f70 0x7fef0807a430 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:08.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.966+0000 7fef2b72b640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef2413c350 0x7fef24154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:08.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.966+0000 7fef2b72b640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef2413bed0 0x7fef24154220 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:08.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.966+0000 7fef2b72b640 1 --2- 172.21.15.98:0/2683005175 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef100063f0 0x7fef24151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:08.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.966+0000 7fef2b72b640 1 -- 172.21.15.98:0/2683005175 >> 172.21.15.98:0/2683005175 conn(0x7fef240b1760 msgr2=0x7fef24158280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:08.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.966+0000 7fef2b72b640 1 -- 172.21.15.98:0/2683005175 shutdown_connections 2024-09-10T21:00:08.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:08.966+0000 7fef2b72b640 1 -- 172.21.15.98:0/2683005175 wait complete. 2024-09-10T21:00:08.973 INFO:tasks.ceph.mds.a.smithi098.stderr: -3158> 2024-09-10T20:59:49.606+0000 7f3e137c8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:00:08.983 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:00:09.005 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-10T21:00:09.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.083+0000 7fecbe575640 1 -- 172.21.15.98:0/2313637621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feca0004800 msgr2=0x7feca0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:09.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.083+0000 7fecbe575640 1 --2- 172.21.15.98:0/2313637621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feca0004800 0x7feca0004c80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fecac006fd0 tx=0x7feca0005830 comp rx=0 tx=0).stop 2024-09-10T21:00:09.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.083+0000 7fecbe575640 1 -- 172.21.15.98:0/2313637621 shutdown_connections 2024-09-10T21:00:09.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.083+0000 7fecbe575640 1 --2- 172.21.15.98:0/2313637621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feca0004800 0x7feca0004c80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:09.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.083+0000 7fecbe575640 1 -- 172.21.15.98:0/2313637621 >> 172.21.15.98:0/2313637621 conn(0x7fecb80d3f70 msgr2=0x7fecb80d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:09.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.083+0000 7fecbe575640 1 -- 172.21.15.98:0/2313637621 shutdown_connections 2024-09-10T21:00:09.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.083+0000 7fecbe575640 1 -- 172.21.15.98:0/2313637621 wait complete. 2024-09-10T21:00:09.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.084+0000 7fecbe575640 1 Processor -- start 2024-09-10T21:00:09.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.084+0000 7fecbe575640 1 -- start start 2024-09-10T21:00:09.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecbe575640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feca0004800 0x7fecb8149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:09.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecbe575640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecb8149fc0 0x7fecb8156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:09.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecbe575640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecb8156880 0x7fecb8154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:09.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecbe575640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fecb80d24e0 con 0x7fecb8156880 2024-09-10T21:00:09.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecbe575640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fecb80b70b0 con 0x7feca0004800 2024-09-10T21:00:09.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecbe575640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fecb80d2660 con 0x7fecb8149fc0 2024-09-10T21:00:09.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecbcb12640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecb8156880 0x7fecb8154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:09.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecbcb12640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecb8156880 0x7fecb8154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:44222/0 (socket says 172.21.15.98:44222) 2024-09-10T21:00:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecbcb12640 1 -- 172.21.15.98:0/3798937062 learned_addr learned my addr 172.21.15.98:0/3798937062 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecb7fff640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feca0004800 0x7fecb8149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecb77fe640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecb8149fc0 0x7fecb8156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecbcb12640 1 -- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecb8149fc0 msgr2=0x7fecb8156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecbcb12640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecb8149fc0 0x7fecb8156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecbcb12640 1 -- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feca0004800 msgr2=0x7fecb8149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecbcb12640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feca0004800 0x7fecb8149a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecbcb12640 1 -- 172.21.15.98:0/3798937062 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb8156f00 con 0x7fecb8156880 2024-09-10T21:00:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.085+0000 7fecb77fe640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecb8149fc0 0x7fecb8156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:00:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.086+0000 7fecbcb12640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecb8156880 0x7fecb8154190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fecb80ce5f0 tx=0x7fecb0006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:09.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.086+0000 7fecb4ff9640 1 -- 172.21.15.98:0/3798937062 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0011840 con 0x7fecb8156880 2024-09-10T21:00:09.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.086+0000 7fecb4ff9640 1 -- 172.21.15.98:0/3798937062 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecb0018070 con 0x7fecb8156880 2024-09-10T21:00:09.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.086+0000 7fecbe575640 1 -- 172.21.15.98:0/3798937062 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb81571e0 con 0x7fecb8156880 2024-09-10T21:00:09.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.086+0000 7fecb4ff9640 1 -- 172.21.15.98:0/3798937062 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0011ca0 con 0x7fecb8156880 2024-09-10T21:00:09.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.086+0000 7fecbe575640 1 -- 172.21.15.98:0/3798937062 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb815fa20 con 0x7fecb8156880 2024-09-10T21:00:09.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.087+0000 7fecbe575640 1 -- 172.21.15.98:0/3798937062 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb80d2b50 con 0x7fecb8156880 2024-09-10T21:00:09.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.087+0000 7fecb4ff9640 1 -- 172.21.15.98:0/3798937062 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fecb001d460 con 0x7fecb8156880 2024-09-10T21:00:09.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.088+0000 7fecb4ff9640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7feca4077f70 0x7feca407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:09.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.088+0000 7fecb4ff9640 1 -- 172.21.15.98:0/3798937062 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7fecb009a7c0 con 0x7fecb8156880 2024-09-10T21:00:09.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.088+0000 7fecb7fff640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7feca4077f70 0x7feca407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:09.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.089+0000 7fecb7fff640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7feca4077f70 0x7feca407a430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fecb8052150 tx=0x7feca8005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:09.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.092+0000 7fecb4ff9640 1 -- 172.21.15.98:0/3798937062 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fecb00630e0 con 0x7fecb8156880 2024-09-10T21:00:09.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.301+0000 7fecbe575640 1 -- 172.21.15.98:0/3798937062 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecb808dfa0 con 0x7fecb8156880 2024-09-10T21:00:09.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.301+0000 7fecb4ff9640 1 -- 172.21.15.98:0/3798937062 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+4398 (secure 0 0 0) 0x7fecb00629e0 con 0x7fecb8156880 2024-09-10T21:00:09.306 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:09.306 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":19,"btime":"2024-09-10T21:00:08:968571+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/1700293059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":1700293059},{"type":"v1","addr":"172.21.15.98:6834","nonce":1700293059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4048376378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4048376378},{"type":"v1","addr":"172.21.15.190:6835","nonce":4048376378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:59:46.344123+0000","modified":"2024-09-10T21:00:08.968557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4834},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4834":{"gid":4834,"name":"b","rank":0,"incarnation":19,"state":"up:replay","state_seq":1,"addr":"172.21.15.190:6837/3087107175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3087107175},{"type":"v1","addr":"172.21.15.190:6837","nonce":3087107175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-10T21:00:09.306 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 19 2024-09-10T21:00:09.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.303+0000 7fecbe575640 1 -- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7feca4077f70 msgr2=0x7feca407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:09.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.303+0000 7fecbe575640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7feca4077f70 0x7feca407a430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fecb8052150 tx=0x7feca8005f90 comp rx=0 tx=0).stop 2024-09-10T21:00:09.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.303+0000 7fecbe575640 1 -- 172.21.15.98:0/3798937062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecb8156880 msgr2=0x7fecb8154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:09.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.303+0000 7fecbe575640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecb8156880 0x7fecb8154190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fecb80ce5f0 tx=0x7fecb0006430 comp rx=0 tx=0).stop 2024-09-10T21:00:09.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.303+0000 7fecbe575640 1 -- 172.21.15.98:0/3798937062 shutdown_connections 2024-09-10T21:00:09.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.303+0000 7fecbe575640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7feca4077f70 0x7feca407a430 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:09.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.303+0000 7fecbe575640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecb8156880 0x7fecb8154190 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:09.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.303+0000 7fecbe575640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fecb8149fc0 0x7fecb8156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:09.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.303+0000 7fecbe575640 1 --2- 172.21.15.98:0/3798937062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feca0004800 0x7fecb8149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:09.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.303+0000 7fecbe575640 1 -- 172.21.15.98:0/3798937062 >> 172.21.15.98:0/3798937062 conn(0x7fecb80d3f70 msgr2=0x7fecb80cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:09.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.304+0000 7fecbe575640 1 -- 172.21.15.98:0/3798937062 shutdown_connections 2024-09-10T21:00:09.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:09.304+0000 7fecbe575640 1 -- 172.21.15.98:0/3798937062 wait complete. 2024-09-10T21:00:09.724 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T20:59:46.344123+0000', 'modified': '2024-09-10T21:00:08.968557+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 41, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4834}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4834': {'gid': 4834, 'name': 'b', 'rank': 0, 'incarnation': 19, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.190:6837/3087107175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 3087107175}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 3087107175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:00:09.724 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4834:up:replay 2024-09-10T21:00:10.726 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:00:10.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.835+0000 7f6021356640 1 -- 172.21.15.98:0/2559873798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f601c15e150 msgr2=0x7f601c15e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.835+0000 7f6021356640 1 --2- 172.21.15.98:0/2559873798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f601c15e150 0x7f601c15e530 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f600c005fd0 tx=0x7f600c02aa90 comp rx=0 tx=0).stop 2024-09-10T21:00:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.835+0000 7f6021356640 1 -- 172.21.15.98:0/2559873798 shutdown_connections 2024-09-10T21:00:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.835+0000 7f6021356640 1 --2- 172.21.15.98:0/2559873798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f601c0cf110 0x7f601c0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.835+0000 7f6021356640 1 --2- 172.21.15.98:0/2559873798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f601c15e150 0x7f601c15e530 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.835+0000 7f6021356640 1 -- 172.21.15.98:0/2559873798 >> 172.21.15.98:0/2559873798 conn(0x7f601c0b1760 msgr2=0x7f601c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.835+0000 7f6021356640 1 -- 172.21.15.98:0/2559873798 shutdown_connections 2024-09-10T21:00:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.835+0000 7f6021356640 1 -- 172.21.15.98:0/2559873798 wait complete. 2024-09-10T21:00:10.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.836+0000 7f6021356640 1 Processor -- start 2024-09-10T21:00:10.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.837+0000 7f6021356640 1 -- start start 2024-09-10T21:00:10.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.837+0000 7f6021356640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f601c06e2e0 0x7f601c1f8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:10.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.837+0000 7f6021356640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f601c0cf110 0x7f601c146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:10.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.837+0000 7f6021356640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f601c15e150 0x7f601c14b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:10.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.837+0000 7f6021356640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f601c0cc570 con 0x7f601c15e150 2024-09-10T21:00:10.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.837+0000 7f6021356640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f601c072a30 con 0x7f601c0cf110 2024-09-10T21:00:10.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.837+0000 7f6021356640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f601c0cc6f0 con 0x7f601c06e2e0 2024-09-10T21:00:10.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.837+0000 7f601b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f601c15e150 0x7f601c14b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:10.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.837+0000 7f601b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f601c15e150 0x7f601c14b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41172/0 (socket says 172.21.15.98:41172) 2024-09-10T21:00:10.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.837+0000 7f601b7fe640 1 -- 172.21.15.98:0/725718250 learned_addr learned my addr 172.21.15.98:0/725718250 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:10.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.837+0000 7f601affd640 1 --2- 172.21.15.98:0/725718250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f601c06e2e0 0x7f601c1f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:10.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.837+0000 7f601a7fc640 1 --2- 172.21.15.98:0/725718250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f601c0cf110 0x7f601c146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:10.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.838+0000 7f601b7fe640 1 -- 172.21.15.98:0/725718250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f601c06e2e0 msgr2=0x7f601c1f8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:10.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.838+0000 7f601b7fe640 1 --2- 172.21.15.98:0/725718250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f601c06e2e0 0x7f601c1f8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:10.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.838+0000 7f601b7fe640 1 -- 172.21.15.98:0/725718250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f601c0cf110 msgr2=0x7f601c146130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:10.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.838+0000 7f601b7fe640 1 --2- 172.21.15.98:0/725718250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f601c0cf110 0x7f601c146130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.838+0000 7f601b7fe640 1 -- 172.21.15.98:0/725718250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c15cc40 con 0x7f601c15e150 2024-09-10T21:00:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.838+0000 7f601b7fe640 1 --2- 172.21.15.98:0/725718250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f601c15e150 0x7f601c14b040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f601400d750 tx=0x7f601400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.838+0000 7f60137fe640 1 -- 172.21.15.98:0/725718250 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014017070 con 0x7f601c15e150 2024-09-10T21:00:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.838+0000 7f60137fe640 1 -- 172.21.15.98:0/725718250 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6014009cc0 con 0x7f601c15e150 2024-09-10T21:00:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.838+0000 7f6021356640 1 -- 172.21.15.98:0/725718250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601c15be80 con 0x7f601c15e150 2024-09-10T21:00:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.838+0000 7f60137fe640 1 -- 172.21.15.98:0/725718250 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601400de60 con 0x7f601c15e150 2024-09-10T21:00:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.838+0000 7f6021356640 1 -- 172.21.15.98:0/725718250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f601c15c440 con 0x7f601c15e150 2024-09-10T21:00:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.839+0000 7f6021356640 1 -- 172.21.15.98:0/725718250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601c0ccba0 con 0x7f601c15e150 2024-09-10T21:00:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.840+0000 7f60137fe640 1 -- 172.21.15.98:0/725718250 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6014005280 con 0x7f601c15e150 2024-09-10T21:00:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.840+0000 7f60137fe640 1 --2- 172.21.15.98:0/725718250 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5ffc077f70 0x7f5ffc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.840+0000 7f60137fe640 1 -- 172.21.15.98:0/725718250 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f6014099f10 con 0x7f601c15e150 2024-09-10T21:00:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.841+0000 7f601affd640 1 --2- 172.21.15.98:0/725718250 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5ffc077f70 0x7f5ffc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.841+0000 7f601affd640 1 --2- 172.21.15.98:0/725718250 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5ffc077f70 0x7f5ffc07a430 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f600c035000 tx=0x7f600c038040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:10.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:10.844+0000 7f60137fe640 1 -- 172.21.15.98:0/725718250 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6014062270 con 0x7f601c15e150 2024-09-10T21:00:11.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:11.060+0000 7f6021356640 1 -- 172.21.15.98:0/725718250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f601c06f630 con 0x7f601c15e150 2024-09-10T21:00:11.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:11.060+0000 7f60137fe640 1 -- 172.21.15.98:0/725718250 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5224 (secure 0 0 0) 0x7f6014066ca0 con 0x7f601c15e150 2024-09-10T21:00:11.064 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:11.065 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":21,"btime":"2024-09-10T21:00:10:972173+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/1700293059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":1700293059},{"type":"v1","addr":"172.21.15.98:6834","nonce":1700293059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4048376378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4048376378},{"type":"v1","addr":"172.21.15.190:6835","nonce":4048376378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/2245019178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2245019178},{"type":"v1","addr":"172.21.15.98:6835","nonce":2245019178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:59:46.344123+0000","modified":"2024-09-10T21:00:09.975424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4834},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4834":{"gid":4834,"name":"b","rank":0,"incarnation":19,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.190:6837/3087107175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3087107175},{"type":"v1","addr":"172.21.15.190:6837","nonce":3087107175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-10T21:00:11.065 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 21 2024-09-10T21:00:11.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:11.062+0000 7f6021356640 1 -- 172.21.15.98:0/725718250 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5ffc077f70 msgr2=0x7f5ffc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:11.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:11.062+0000 7f6021356640 1 --2- 172.21.15.98:0/725718250 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5ffc077f70 0x7f5ffc07a430 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f600c035000 tx=0x7f600c038040 comp rx=0 tx=0).stop 2024-09-10T21:00:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:11.062+0000 7f6021356640 1 -- 172.21.15.98:0/725718250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f601c15e150 msgr2=0x7f601c14b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:11.062+0000 7f6021356640 1 --2- 172.21.15.98:0/725718250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f601c15e150 0x7f601c14b040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f601400d750 tx=0x7f601400dc10 comp rx=0 tx=0).stop 2024-09-10T21:00:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:11.062+0000 7f6021356640 1 -- 172.21.15.98:0/725718250 shutdown_connections 2024-09-10T21:00:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:11.062+0000 7f6021356640 1 --2- 172.21.15.98:0/725718250 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f5ffc077f70 0x7f5ffc07a430 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:11.062+0000 7f6021356640 1 --2- 172.21.15.98:0/725718250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f601c15e150 0x7f601c14b040 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:11.062+0000 7f6021356640 1 --2- 172.21.15.98:0/725718250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f601c0cf110 0x7f601c146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:11.062+0000 7f6021356640 1 --2- 172.21.15.98:0/725718250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f601c06e2e0 0x7f601c1f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:11.062+0000 7f6021356640 1 -- 172.21.15.98:0/725718250 >> 172.21.15.98:0/725718250 conn(0x7f601c0b1760 msgr2=0x7f601c15e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:11.062+0000 7f6021356640 1 -- 172.21.15.98:0/725718250 shutdown_connections 2024-09-10T21:00:11.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:11.062+0000 7f6021356640 1 -- 172.21.15.98:0/725718250 wait complete. 2024-09-10T21:00:11.480 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T20:59:46.344123+0000', 'modified': '2024-09-10T21:00:09.975424+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 41, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4834}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4834': {'gid': 4834, 'name': 'b', 'rank': 0, 'incarnation': 19, 'state': 'up:rejoin', 'state_seq': 10, 'addr': '172.21.15.190:6837/3087107175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 3087107175}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 3087107175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:00:11.480 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4834:up:rejoin 2024-09-10T21:00:12.481 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:00:12.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.590+0000 7ff96d7d3640 1 -- 172.21.15.98:0/1274932408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff96806d160 msgr2=0x7ff96806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:12.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.590+0000 7ff96d7d3640 1 --2- 172.21.15.98:0/1274932408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff96806d160 0x7ff96806fc90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff968075210 tx=0x7ff95802a4d0 comp rx=0 tx=0).stop 2024-09-10T21:00:12.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.590+0000 7ff96d7d3640 1 -- 172.21.15.98:0/1274932408 shutdown_connections 2024-09-10T21:00:12.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.590+0000 7ff96d7d3640 1 --2- 172.21.15.98:0/1274932408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff96806d160 0x7ff96806fc90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:12.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.590+0000 7ff96d7d3640 1 --2- 172.21.15.98:0/1274932408 >> v2:172.21.15.190:3300/0 conn(0x7ff968152d00 0x7ff968160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:12.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.590+0000 7ff96d7d3640 1 -- 172.21.15.98:0/1274932408 >> 172.21.15.98:0/1274932408 conn(0x7ff9680b2770 msgr2=0x7ff9680b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:12.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.590+0000 7ff96d7d3640 1 -- 172.21.15.98:0/1274932408 shutdown_connections 2024-09-10T21:00:12.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.590+0000 7ff96d7d3640 1 -- 172.21.15.98:0/1274932408 wait complete. 2024-09-10T21:00:12.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.592+0000 7ff96d7d3640 1 Processor -- start 2024-09-10T21:00:12.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.592+0000 7ff96d7d3640 1 -- start start 2024-09-10T21:00:12.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff96d7d3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff96806d160 0x7ff968147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:12.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff96d7d3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9680cee50 0x7ff968150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:12.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff96d7d3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff968152d00 0x7ff96815e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:12.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff96d7d3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff968070350 con 0x7ff968152d00 2024-09-10T21:00:12.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff96d7d3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff9680701d0 con 0x7ff96806d160 2024-09-10T21:00:12.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff96d7d3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff9680704d0 con 0x7ff9680cee50 2024-09-10T21:00:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff9677fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff968152d00 0x7ff96815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff9677fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff968152d00 0x7ff96815e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41190/0 (socket says 172.21.15.98:41190) 2024-09-10T21:00:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff9677fe640 1 -- 172.21.15.98:0/885877623 learned_addr learned my addr 172.21.15.98:0/885877623 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff9667fc640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9680cee50 0x7ff968150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff966ffd640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff96806d160 0x7ff968147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff9677fe640 1 -- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9680cee50 msgr2=0x7ff968150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff9677fe640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9680cee50 0x7ff968150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff9677fe640 1 -- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff96806d160 msgr2=0x7ff968147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff9677fe640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff96806d160 0x7ff968147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff9677fe640 1 -- 172.21.15.98:0/885877623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9681f6430 con 0x7ff968152d00 2024-09-10T21:00:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff9667fc640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9680cee50 0x7ff968150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff9677fe640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff968152d00 0x7ff96815e570 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff9580099c0 tx=0x7ff958003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff95f7fe640 1 -- 172.21.15.98:0/885877623 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95803f070 con 0x7ff968152d00 2024-09-10T21:00:12.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff95f7fe640 1 -- 172.21.15.98:0/885877623 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff958004890 con 0x7ff968152d00 2024-09-10T21:00:12.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff96d7d3640 1 -- 172.21.15.98:0/885877623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff968147a40 con 0x7ff968152d00 2024-09-10T21:00:12.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff95f7fe640 1 -- 172.21.15.98:0/885877623 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958035030 con 0x7ff968152d00 2024-09-10T21:00:12.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.593+0000 7ff96d7d3640 1 -- 172.21.15.98:0/885877623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9681f5b90 con 0x7ff968152d00 2024-09-10T21:00:12.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.594+0000 7ff96d7d3640 1 -- 172.21.15.98:0/885877623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9680709f0 con 0x7ff968152d00 2024-09-10T21:00:12.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.595+0000 7ff95f7fe640 1 -- 172.21.15.98:0/885877623 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff958004ce0 con 0x7ff968152d00 2024-09-10T21:00:12.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.596+0000 7ff95f7fe640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff948077f70 0x7ff94807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:12.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.596+0000 7ff95f7fe640 1 -- 172.21.15.98:0/885877623 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7ff9580bb140 con 0x7ff968152d00 2024-09-10T21:00:12.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.596+0000 7ff966ffd640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff948077f70 0x7ff94807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:12.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.597+0000 7ff966ffd640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff948077f70 0x7ff94807a430 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff96000a2e0 tx=0x7ff96000a270 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:12.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.600+0000 7ff95f7fe640 1 -- 172.21.15.98:0/885877623 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff9681605b0 con 0x7ff968152d00 2024-09-10T21:00:12.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.808+0000 7ff96d7d3640 1 -- 172.21.15.98:0/885877623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff968150950 con 0x7ff968152d00 2024-09-10T21:00:12.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.808+0000 7ff95f7fe640 1 -- 172.21.15.98:0/885877623 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5230 (secure 0 0 0) 0x7ff958083bb0 con 0x7ff968152d00 2024-09-10T21:00:12.812 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:12.812 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":22,"btime":"2024-09-10T21:00:11:973417+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/1700293059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":1700293059},{"type":"v1","addr":"172.21.15.98:6834","nonce":1700293059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4048376378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4048376378},{"type":"v1","addr":"172.21.15.190:6835","nonce":4048376378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/2245019178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2245019178},{"type":"v1","addr":"172.21.15.98:6835","nonce":2245019178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:59:46.344123+0000","modified":"2024-09-10T21:00:10.978422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4834},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4834":{"gid":4834,"name":"b","rank":0,"incarnation":19,"state":"up:clientreplay","state_seq":11,"addr":"172.21.15.190:6837/3087107175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3087107175},{"type":"v1","addr":"172.21.15.190:6837","nonce":3087107175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-10T21:00:12.813 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 22 2024-09-10T21:00:12.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.810+0000 7ff96d7d3640 1 -- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff948077f70 msgr2=0x7ff94807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:12.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.810+0000 7ff96d7d3640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff948077f70 0x7ff94807a430 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff96000a2e0 tx=0x7ff96000a270 comp rx=0 tx=0).stop 2024-09-10T21:00:12.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.810+0000 7ff96d7d3640 1 -- 172.21.15.98:0/885877623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff968152d00 msgr2=0x7ff96815e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:12.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.810+0000 7ff96d7d3640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff968152d00 0x7ff96815e570 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff9580099c0 tx=0x7ff958003960 comp rx=0 tx=0).stop 2024-09-10T21:00:12.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.810+0000 7ff96d7d3640 1 -- 172.21.15.98:0/885877623 shutdown_connections 2024-09-10T21:00:12.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.810+0000 7ff96d7d3640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff948077f70 0x7ff94807a430 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:12.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.810+0000 7ff96d7d3640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff968152d00 0x7ff96815e570 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:12.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.810+0000 7ff96d7d3640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9680cee50 0x7ff968150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:12.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.810+0000 7ff96d7d3640 1 --2- 172.21.15.98:0/885877623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff96806d160 0x7ff968147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:12.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.810+0000 7ff96d7d3640 1 -- 172.21.15.98:0/885877623 >> 172.21.15.98:0/885877623 conn(0x7ff9680b2770 msgr2=0x7ff96815d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:12.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.811+0000 7ff96d7d3640 1 -- 172.21.15.98:0/885877623 shutdown_connections 2024-09-10T21:00:12.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:12.811+0000 7ff96d7d3640 1 -- 172.21.15.98:0/885877623 wait complete. 2024-09-10T21:00:13.228 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T20:59:46.344123+0000', 'modified': '2024-09-10T21:00:10.978422+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 41, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4834}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4834': {'gid': 4834, 'name': 'b', 'rank': 0, 'incarnation': 19, 'state': 'up:clientreplay', 'state_seq': 11, 'addr': '172.21.15.190:6837/3087107175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 3087107175}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 3087107175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:00:13.228 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4834:up:clientreplay 2024-09-10T21:00:14.229 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:00:14.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.339+0000 7f07f8b7c640 1 -- 172.21.15.98:0/3793930900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e00040b0 msgr2=0x7f07e0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:14.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.339+0000 7f07f8b7c640 1 --2- 172.21.15.98:0/3793930900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e00040b0 0x7f07e0004800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f07f4075550 tx=0x7f07e0005830 comp rx=0 tx=0).stop 2024-09-10T21:00:14.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.339+0000 7f07f2d9d640 1 -- 172.21.15.98:0/3793930900 reap_dead start 2024-09-10T21:00:14.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.339+0000 7f07f8b7c640 1 -- 172.21.15.98:0/3793930900 shutdown_connections 2024-09-10T21:00:14.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.339+0000 7f07f8b7c640 1 -- 172.21.15.98:0/3793930900 >> 172.21.15.98:0/3793930900 conn(0x7f07f40d1d10 msgr2=0x7f07f40d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:14.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.339+0000 7f07f8b7c640 1 -- 172.21.15.98:0/3793930900 shutdown_connections 2024-09-10T21:00:14.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.339+0000 7f07f8b7c640 1 -- 172.21.15.98:0/3793930900 wait complete. 2024-09-10T21:00:14.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.341+0000 7f07f8b7c640 1 Processor -- start 2024-09-10T21:00:14.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.341+0000 7f07f8b7c640 1 -- start start 2024-09-10T21:00:14.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f8b7c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07e00040b0 0x7f07f41f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:14.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f8b7c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07f40caae0 0x7f07f40b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:14.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f8b7c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07f40b6fa0 0x7f07f40afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:14.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f8b7c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f07f40cf8e0 con 0x7f07f40b6fa0 2024-09-10T21:00:14.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f8b7c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f07f4070a50 con 0x7f07e00040b0 2024-09-10T21:00:14.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f8b7c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f07f40cfa60 con 0x7f07f40caae0 2024-09-10T21:00:14.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07f40b6fa0 0x7f07f40afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:14.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07f40b6fa0 0x7f07f40afb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41212/0 (socket says 172.21.15.98:41212) 2024-09-10T21:00:14.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f2d9d640 1 -- 172.21.15.98:0/4281322392 learned_addr learned my addr 172.21.15.98:0/4281322392 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f259c640 1 --2- 172.21.15.98:0/4281322392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07e00040b0 0x7f07f41f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07ebfff640 1 --2- 172.21.15.98:0/4281322392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07f40caae0 0x7f07f40b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f2d9d640 1 -- 172.21.15.98:0/4281322392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07f40caae0 msgr2=0x7f07f40b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f2d9d640 1 --2- 172.21.15.98:0/4281322392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07f40caae0 0x7f07f40b6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f2d9d640 1 -- 172.21.15.98:0/4281322392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07e00040b0 msgr2=0x7f07f41f2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f2d9d640 1 --2- 172.21.15.98:0/4281322392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07e00040b0 0x7f07f41f2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f2d9d640 1 -- 172.21.15.98:0/4281322392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f40b5960 con 0x7f07f40b6fa0 2024-09-10T21:00:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f2d9d640 1 --2- 172.21.15.98:0/4281322392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07f40b6fa0 0x7f07f40afb40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f07f40734b0 tx=0x7f07ec0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07eaffd640 1 -- 172.21.15.98:0/4281322392 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec011870 con 0x7f07f40b6fa0 2024-09-10T21:00:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07eaffd640 1 -- 172.21.15.98:0/4281322392 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07ec017070 con 0x7f07f40b6fa0 2024-09-10T21:00:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f8b7c640 1 -- 172.21.15.98:0/4281322392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f41f2c40 con 0x7f07f40b6fa0 2024-09-10T21:00:14.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07eaffd640 1 -- 172.21.15.98:0/4281322392 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec011d60 con 0x7f07f40b6fa0 2024-09-10T21:00:14.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.342+0000 7f07f8b7c640 1 -- 172.21.15.98:0/4281322392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f40b5030 con 0x7f07f40b6fa0 2024-09-10T21:00:14.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.344+0000 7f07f8b7c640 1 -- 172.21.15.98:0/4281322392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f41f5cf0 con 0x7f07f40b6fa0 2024-09-10T21:00:14.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.344+0000 7f07eaffd640 1 -- 172.21.15.98:0/4281322392 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f07ec01c460 con 0x7f07f40b6fa0 2024-09-10T21:00:14.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.344+0000 7f07eaffd640 1 --2- 172.21.15.98:0/4281322392 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f07d4077f70 0x7f07d407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:14.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.345+0000 7f07eaffd640 1 -- 172.21.15.98:0/4281322392 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f07ec09a6f0 con 0x7f07f40b6fa0 2024-09-10T21:00:14.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.345+0000 7f07f259c640 1 --2- 172.21.15.98:0/4281322392 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f07d4077f70 0x7f07d407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:14.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.345+0000 7f07f259c640 1 --2- 172.21.15.98:0/4281322392 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f07d4077f70 0x7f07d407a430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f07e4001010 tx=0x7f07e40067a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:14.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.349+0000 7f07eaffd640 1 -- 172.21.15.98:0/4281322392 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f07ec0629c0 con 0x7f07f40b6fa0 2024-09-10T21:00:14.394 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:00:14.397 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:00:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.562+0000 7f07f8b7c640 1 -- 172.21.15.98:0/4281322392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07f40b7c90 con 0x7f07f40b6fa0 2024-09-10T21:00:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.563+0000 7f07eaffd640 1 -- 172.21.15.98:0/4281322392 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5231 (secure 0 0 0) 0x7f07ec0673f0 con 0x7f07f40b6fa0 2024-09-10T21:00:14.567 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:14.567 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":23,"btime":"2024-09-10T21:00:12:974795+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/1700293059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":1700293059},{"type":"v1","addr":"172.21.15.98:6834","nonce":1700293059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4048376378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4048376378},{"type":"v1","addr":"172.21.15.190:6835","nonce":4048376378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/2245019178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2245019178},{"type":"v1","addr":"172.21.15.98:6835","nonce":2245019178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:59:46.344123+0000","modified":"2024-09-10T21:00:12.974787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4834},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4834":{"gid":4834,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.190:6837/3087107175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3087107175},{"type":"v1","addr":"172.21.15.190:6837","nonce":3087107175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4834,"qdb_cluster":[4834]},"id":2}]} 2024-09-10T21:00:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 23 2024-09-10T21:00:14.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.565+0000 7f07f8b7c640 1 -- 172.21.15.98:0/4281322392 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f07d4077f70 msgr2=0x7f07d407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:14.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.565+0000 7f07f8b7c640 1 --2- 172.21.15.98:0/4281322392 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f07d4077f70 0x7f07d407a430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f07e4001010 tx=0x7f07e40067a0 comp rx=0 tx=0).stop 2024-09-10T21:00:14.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.565+0000 7f07f8b7c640 1 -- 172.21.15.98:0/4281322392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07f40b6fa0 msgr2=0x7f07f40afb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:14.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.565+0000 7f07f8b7c640 1 --2- 172.21.15.98:0/4281322392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07f40b6fa0 0x7f07f40afb40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f07f40734b0 tx=0x7f07ec0019b0 comp rx=0 tx=0).stop 2024-09-10T21:00:14.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.565+0000 7f07f8b7c640 1 -- 172.21.15.98:0/4281322392 shutdown_connections 2024-09-10T21:00:14.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.565+0000 7f07f8b7c640 1 --2- 172.21.15.98:0/4281322392 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f07d4077f70 0x7f07d407a430 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:14.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.565+0000 7f07f8b7c640 1 --2- 172.21.15.98:0/4281322392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07f40b6fa0 0x7f07f40afb40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:14.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.565+0000 7f07f8b7c640 1 --2- 172.21.15.98:0/4281322392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07f40caae0 0x7f07f40b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:14.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.565+0000 7f07f8b7c640 1 --2- 172.21.15.98:0/4281322392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07e00040b0 0x7f07f41f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:14.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.565+0000 7f07f8b7c640 1 -- 172.21.15.98:0/4281322392 >> 172.21.15.98:0/4281322392 conn(0x7f07f40d1d10 msgr2=0x7f07f4159b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:14.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.565+0000 7f07f8b7c640 1 -- 172.21.15.98:0/4281322392 shutdown_connections 2024-09-10T21:00:14.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:14.565+0000 7f07f8b7c640 1 -- 172.21.15.98:0/4281322392 wait complete. 2024-09-10T21:00:14.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T20:59:46.344123+0000', 'modified': '2024-09-10T21:00:12.974787+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 41, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4834}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4834': {'gid': 4834, 'name': 'b', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.190:6837/3087107175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 3087107175}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 3087107175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4834, 'qdb_cluster': [4834]} 2024-09-10T21:00:14.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:00:14.985 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T21:00:14.991 INFO:tasks.ceph.mds.b.smithi190.stderr:2024-09-10T21:00:14.987+0000 7f78b7a08640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:00:15.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.084+0000 7f9bdec9b640 1 -- 172.21.15.98:0/2604489577 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bc0004d40 msgr2=0x7f9bc00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.084+0000 7f9bdec9b640 1 --2- 172.21.15.98:0/2604489577 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bc0004d40 0x7f9bc00051c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000c260 tx=0x7f9bc0005830 comp rx=0 tx=0).stop 2024-09-10T21:00:15.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.084+0000 7f9bdd238640 1 -- 172.21.15.98:0/2604489577 reap_dead start 2024-09-10T21:00:15.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.084+0000 7f9bdec9b640 1 -- 172.21.15.98:0/2604489577 shutdown_connections 2024-09-10T21:00:15.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.084+0000 7f9bdec9b640 1 -- 172.21.15.98:0/2604489577 >> 172.21.15.98:0/2604489577 conn(0x7f9bd80b1760 msgr2=0x7f9bd80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:15.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.084+0000 7f9bdec9b640 1 -- 172.21.15.98:0/2604489577 shutdown_connections 2024-09-10T21:00:15.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.084+0000 7f9bdec9b640 1 -- 172.21.15.98:0/2604489577 wait complete. 2024-09-10T21:00:15.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.085+0000 7f9bdec9b640 1 Processor -- start 2024-09-10T21:00:15.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.085+0000 7f9bdec9b640 1 -- start start 2024-09-10T21:00:15.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.086+0000 7f9bdec9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bc0004d40 0x7f9bd813e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.086+0000 7f9bdec9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bd815af40 0x7f9bd81f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.086+0000 7f9bdec9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd81f4ce0 0x7f9bd80afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.086+0000 7f9bdec9b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd815cf70 con 0x7f9bd81f4ce0 2024-09-10T21:00:15.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.086+0000 7f9bdec9b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd815cdf0 con 0x7f9bd815af40 2024-09-10T21:00:15.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.086+0000 7f9bdec9b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd8070150 con 0x7f9bc0004d40 2024-09-10T21:00:15.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.086+0000 7f9bdd238640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd81f4ce0 0x7f9bd80afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.086+0000 7f9bdd238640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd81f4ce0 0x7f9bd80afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41240/0 (socket says 172.21.15.98:41240) 2024-09-10T21:00:15.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.086+0000 7f9bdd238640 1 -- 172.21.15.98:0/868047884 learned_addr learned my addr 172.21.15.98:0/868047884 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:15.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.086+0000 7f9bdca37640 1 --2- 172.21.15.98:0/868047884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bc0004d40 0x7f9bd813e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.086+0000 7f9bcffff640 1 --2- 172.21.15.98:0/868047884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bd815af40 0x7f9bd81f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.087+0000 7f9bdd238640 1 -- 172.21.15.98:0/868047884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bc0004d40 msgr2=0x7f9bd813e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.087+0000 7f9bdd238640 1 --2- 172.21.15.98:0/868047884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bc0004d40 0x7f9bd813e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.087+0000 7f9bdd238640 1 -- 172.21.15.98:0/868047884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bd815af40 msgr2=0x7f9bd81f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.087+0000 7f9bdd238640 1 --2- 172.21.15.98:0/868047884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bd815af40 0x7f9bd81f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.087+0000 7f9bdd238640 1 -- 172.21.15.98:0/868047884 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd81f52d0 con 0x7f9bd81f4ce0 2024-09-10T21:00:15.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.087+0000 7f9bdd238640 1 --2- 172.21.15.98:0/868047884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd81f4ce0 0x7f9bd80afe90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000c380 tx=0x7f9bd0006650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:15.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.087+0000 7f9bcd7fa640 1 -- 172.21.15.98:0/868047884 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0031070 con 0x7f9bd81f4ce0 2024-09-10T21:00:15.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.087+0000 7f9bcd7fa640 1 -- 172.21.15.98:0/868047884 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bd002c680 con 0x7f9bd81f4ce0 2024-09-10T21:00:15.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.087+0000 7f9bcd7fa640 1 -- 172.21.15.98:0/868047884 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0006c50 con 0x7f9bd81f4ce0 2024-09-10T21:00:15.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.087+0000 7f9bdec9b640 1 -- 172.21.15.98:0/868047884 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd81f55b0 con 0x7f9bd81f4ce0 2024-09-10T21:00:15.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.087+0000 7f9bdec9b640 1 -- 172.21.15.98:0/868047884 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd815aa20 con 0x7f9bd81f4ce0 2024-09-10T21:00:15.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.089+0000 7f9bcd7fa640 1 -- 172.21.15.98:0/868047884 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9bd002ca70 con 0x7f9bd81f4ce0 2024-09-10T21:00:15.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.089+0000 7f9bcd7fa640 1 --2- 172.21.15.98:0/868047884 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9bc4077f70 0x7f9bc407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.089+0000 7f9bcd7fa640 1 -- 172.21.15.98:0/868047884 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f9bd00b9970 con 0x7f9bd81f4ce0 2024-09-10T21:00:15.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.089+0000 7f9bdca37640 1 --2- 172.21.15.98:0/868047884 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9bc4077f70 0x7f9bc407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.090+0000 7f9be1a6a740 1 Processor -- start 2024-09-10T21:00:15.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.090+0000 7f9be1a6a740 1 -- start start 2024-09-10T21:00:15.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.090+0000 7f9bdca37640 1 --2- 172.21.15.98:0/868047884 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9bc4077f70 0x7f9bc407a430 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4005fd0 tx=0x7f9bd4009660 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:15.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.090+0000 7f9be1a6a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cd9d1b0 0x56016cd74b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.090+0000 7f9be1a6a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cdbcdf0 0x56016cdbd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.090+0000 7f9be1a6a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd725f0 0x56016cd72a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.090+0000 7f9be1a6a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56016cce9820 con 0x56016cdbcdf0 2024-09-10T21:00:15.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.090+0000 7f9be1a6a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56016ccd8c50 con 0x56016cd9d1b0 2024-09-10T21:00:15.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.090+0000 7f9be1a6a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x56016ccdca60 con 0x56016cd725f0 2024-09-10T21:00:15.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cdbcdf0 0x56016cdbd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cdbcdf0 0x56016cdbd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41254/0 (socket says 172.21.15.98:41254) 2024-09-10T21:00:15.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bcffff640 1 -- 172.21.15.98:0/2671293824 learned_addr learned my addr 172.21.15.98:0/2671293824 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:15.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bdca37640 1 --2- 172.21.15.98:0/2671293824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cd9d1b0 0x56016cd74b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bdd238640 1 --2- 172.21.15.98:0/2671293824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd725f0 0x56016cd72a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bcffff640 1 -- 172.21.15.98:0/2671293824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd725f0 msgr2=0x56016cd72a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:00:15.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bcffff640 1 --2- 172.21.15.98:0/2671293824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd725f0 0x56016cd72a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bcffff640 1 -- 172.21.15.98:0/2671293824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cd9d1b0 msgr2=0x56016cd74b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:00:15.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bcffff640 1 --2- 172.21.15.98:0/2671293824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cd9d1b0 0x56016cd74b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bcffff640 1 -- 172.21.15.98:0/2671293824 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56016ca3a850 con 0x56016cdbcdf0 2024-09-10T21:00:15.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bdca37640 1 --2- 172.21.15.98:0/2671293824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cd9d1b0 0x56016cd74b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:00:15.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bcffff640 1 --2- 172.21.15.98:0/2671293824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cdbcdf0 0x56016cdbd1f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800d710 tx=0x7f9bc800dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15b3144ee94ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:15.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bbe7fc640 1 -- 172.21.15.98:0/2671293824 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8003dd0 con 0x56016cdbcdf0 2024-09-10T21:00:15.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bbe7fc640 1 -- 172.21.15.98:0/2671293824 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bc801b440 con 0x56016cdbcdf0 2024-09-10T21:00:15.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.091+0000 7f9bbe7fc640 1 -- 172.21.15.98:0/2671293824 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc801b890 con 0x56016cdbcdf0 2024-09-10T21:00:15.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.092+0000 7f9be1a6a740 1 -- 172.21.15.98:0/2671293824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cdbcdf0 msgr2=0x56016cdbd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.092+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/2671293824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cdbcdf0 0x56016cdbd1f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800d710 tx=0x7f9bc800dbd0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.092+0000 7f9be1a6a740 1 -- 172.21.15.98:0/2671293824 shutdown_connections 2024-09-10T21:00:15.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.092+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/2671293824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd725f0 0x56016cd72a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.092+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/2671293824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cdbcdf0 0x56016cdbd1f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.092+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/2671293824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cd9d1b0 0x56016cd74b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.092+0000 7f9be1a6a740 1 -- 172.21.15.98:0/2671293824 >> 172.21.15.98:0/2671293824 conn(0x56016cd736a0 msgr2=0x56016cd770c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:15.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.092+0000 7f9be1a6a740 1 -- 172.21.15.98:0/2671293824 shutdown_connections 2024-09-10T21:00:15.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.092+0000 7f9be1a6a740 1 -- 172.21.15.98:0/2671293824 wait complete. 2024-09-10T21:00:15.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.092+0000 7f9be1a6a740 1 -- start start 2024-09-10T21:00:15.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.093+0000 7f9be1a6a740 1 Processor -- start 2024-09-10T21:00:15.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.093+0000 7f9be1a6a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cdb5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.093+0000 7f9be1a6a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cdbcdf0 0x56016cd94a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.093+0000 7f9be1a6a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd94fb0 0x56016cdcb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.093+0000 7f9be1a6a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56016ca49620 con 0x56016cd9d1b0 2024-09-10T21:00:15.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.093+0000 7f9be1a6a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56016cc7e5f0 con 0x56016cdbcdf0 2024-09-10T21:00:15.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.093+0000 7f9be1a6a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x56016cd71b30 con 0x56016cd94fb0 2024-09-10T21:00:15.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.093+0000 7f9be1a6a740 10 client.0 _ll_register_callbacks cb 0x56016cc820a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:00:15.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.093+0000 7f9bdca37640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cdb5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.093+0000 7f9bdca37640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cdb5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41256/0 (socket says 172.21.15.98:41256) 2024-09-10T21:00:15.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.093+0000 7f9bdca37640 1 -- 172.21.15.98:0/59035301 learned_addr learned my addr 172.21.15.98:0/59035301 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:15.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bcffff640 1 --2- 172.21.15.98:0/59035301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cdbcdf0 0x56016cd94a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bdd238640 1 --2- 172.21.15.98:0/59035301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd94fb0 0x56016cdcb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bdca37640 1 -- 172.21.15.98:0/59035301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd94fb0 msgr2=0x56016cdcb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:00:15.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bdca37640 1 --2- 172.21.15.98:0/59035301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd94fb0 0x56016cdcb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bdca37640 1 -- 172.21.15.98:0/59035301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cdbcdf0 msgr2=0x56016cd94a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:00:15.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bdca37640 1 --2- 172.21.15.98:0/59035301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cdbcdf0 0x56016cd94a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bdca37640 1 -- 172.21.15.98:0/59035301 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56016cab16b0 con 0x56016cd9d1b0 2024-09-10T21:00:15.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bcffff640 1 --2- 172.21.15.98:0/59035301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cdbcdf0 0x56016cd94a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:00:15.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bdca37640 1 --2- 172.21.15.98:0/59035301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cdb5170 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9bd401d460 tx=0x7f9bd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14ac9f924fd2a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:15.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-10T21:00:15.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bb77fe640 1 -- 172.21.15.98:0/59035301 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd402b060 con 0x56016cd9d1b0 2024-09-10T21:00:15.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bb77fe640 1 -- 172.21.15.98:0/59035301 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f9bd401b660 con 0x56016cd9d1b0 2024-09-10T21:00:15.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9be1a6a740 1 -- 172.21.15.98:0/59035301 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc801b440 con 0x56016cd9d1b0 2024-09-10T21:00:15.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bb77fe640 1 -- 172.21.15.98:0/59035301 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9bd4004b20 con 0x56016cd9d1b0 2024-09-10T21:00:15.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9be1a6a740 10 client.5265 fetch_fsmap learned FSMap version 23 2024-09-10T21:00:15.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9be1a6a740 1 -- 172.21.15.98:0/59035301 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x56016cdb2bf0 con 0x56016cd9d1b0 2024-09-10T21:00:15.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bb77fe640 1 -- 172.21.15.98:0/59035301 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bc801b440 con 0x56016cd9d1b0 2024-09-10T21:00:15.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9bb77fe640 1 -- 172.21.15.98:0/59035301 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f9bd4034080 con 0x56016cd9d1b0 2024-09-10T21:00:15.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.094+0000 7f9be1a6a740 10 client.5265 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T21:00:15.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.095+0000 7f9be1a6a740 10 client.5265 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:00:15.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.095+0000 7f9be1a6a740 20 client.5265 populate_metadata read hostname 'smithi098' 2024-09-10T21:00:15.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.095+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/59035301 >> [v2:172.21.15.190:6836/3087107175,v1:172.21.15.190:6837/3087107175] conn(0x56016cdceed0 0x56016cdb3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.095+0000 7f9be1a6a740 4 client.5265 mds_command: new command op to 4834 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:00:15.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.095+0000 7f9be1a6a740 1 -- 172.21.15.98:0/59035301 --> [v2:172.21.15.190:6836/3087107175,v1:172.21.15.190:6837/3087107175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56016cceb770 con 0x56016cdceed0 2024-09-10T21:00:15.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.095+0000 7f9bcffff640 1 --2- 172.21.15.98:0/59035301 >> [v2:172.21.15.190:6836/3087107175,v1:172.21.15.190:6837/3087107175] conn(0x56016cdceed0 0x56016cdb3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.095+0000 7f9bb77fe640 10 client.5265 ms_handle_connect on v2:172.21.15.190:6836/3087107175 2024-09-10T21:00:15.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.095+0000 7f9bcffff640 1 --2- 172.21.15.98:0/59035301 >> [v2:172.21.15.190:6836/3087107175,v1:172.21.15.190:6837/3087107175] conn(0x56016cdceed0 0x56016cdb3260 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75be40aa6417b1ce server_cookie=c2980f8b0febf746 in_seq=0 out_seq=0 2024-09-10T21:00:15.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9bb77fe640 1 -- 172.21.15.98:0/59035301 <== mds.0 v2:172.21.15.190:6836/3087107175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56016cceb770 con 0x56016cdceed0 2024-09-10T21:00:15.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9bb77fe640 10 client.5265 handle_command_reply: tid=0 2024-09-10T21:00:15.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9be1a6a740 1 client.5265 shutdown 2024-09-10T21:00:15.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9be1a6a740 1 -- 172.21.15.98:0/59035301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 msgr2=0x56016cdb5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/59035301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cdb5170 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9bd401d460 tx=0x7f9bd4004a40 comp rx=0 tx=0).stop 2024-09-10T21:00:15.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9be1a6a740 1 -- 172.21.15.98:0/59035301 shutdown_connections 2024-09-10T21:00:15.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/59035301 >> [v2:172.21.15.190:6836/3087107175,v1:172.21.15.190:6837/3087107175] conn(0x56016cdceed0 0x56016cdb3260 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/59035301 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd94fb0 0x56016cdcb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9bb77fe640 0 client.5265 ms_handle_reset on v2:172.21.15.190:6836/3087107175 2024-09-10T21:00:15.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/59035301 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cdbcdf0 0x56016cd94a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/59035301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cdb5170 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9be1a6a740 1 -- 172.21.15.98:0/59035301 >> 172.21.15.98:0/59035301 conn(0x56016cd725f0 msgr2=0x56016cd76860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:15.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9be1a6a740 1 -- 172.21.15.98:0/59035301 shutdown_connections 2024-09-10T21:00:15.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9be1a6a740 1 -- 172.21.15.98:0/59035301 wait complete. 2024-09-10T21:00:15.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.096+0000 7f9be1a6a740 20 client.5265 trim_cache size 0 max 16384 2024-09-10T21:00:15.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.114+0000 7f9be1a6a740 1 Processor -- start 2024-09-10T21:00:15.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.114+0000 7f9be1a6a740 1 -- start start 2024-09-10T21:00:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9be1a6a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cd94fb0 0x56016cd707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9be1a6a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cddf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9be1a6a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cdbcdf0 0x56016cddf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9be1a6a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8200eb0 con 0x56016cd9d1b0 2024-09-10T21:00:15.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9be1a6a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8200210 con 0x56016cd94fb0 2024-09-10T21:00:15.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9be1a6a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd8200860 con 0x56016cdbcdf0 2024-09-10T21:00:15.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cddf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cddf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41264/0 (socket says 172.21.15.98:41264) 2024-09-10T21:00:15.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9bcffff640 1 -- 172.21.15.98:0/3068025568 learned_addr learned my addr 172.21.15.98:0/3068025568 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:15.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9bcffff640 1 -- 172.21.15.98:0/3068025568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cdbcdf0 msgr2=0x56016cddf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:00:15.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9bcffff640 1 --2- 172.21.15.98:0/3068025568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cdbcdf0 0x56016cddf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9bcffff640 1 -- 172.21.15.98:0/3068025568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cd94fb0 msgr2=0x56016cd707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:00:15.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9bcffff640 1 --2- 172.21.15.98:0/3068025568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cd94fb0 0x56016cd707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9bcffff640 1 -- 172.21.15.98:0/3068025568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56016cd74430 con 0x56016cd9d1b0 2024-09-10T21:00:15.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9bcffff640 1 --2- 172.21.15.98:0/3068025568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cddf1f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x56016cdbdc30 tx=0x7f9bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898bc574c87409ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:15.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.115+0000 7f9bbdffb640 1 -- 172.21.15.98:0/3068025568 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80049a0 con 0x56016cd9d1b0 2024-09-10T21:00:15.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.116+0000 7f9bbdffb640 1 -- 172.21.15.98:0/3068025568 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bc8004c30 con 0x56016cd9d1b0 2024-09-10T21:00:15.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.116+0000 7f9bbdffb640 1 -- 172.21.15.98:0/3068025568 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8005290 con 0x56016cd9d1b0 2024-09-10T21:00:15.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.116+0000 7f9be1a6a740 1 -- 172.21.15.98:0/3068025568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 msgr2=0x56016cddf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.116+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/3068025568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cddf1f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x56016cdbdc30 tx=0x7f9bc80046c0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.116+0000 7f9be1a6a740 1 -- 172.21.15.98:0/3068025568 shutdown_connections 2024-09-10T21:00:15.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.116+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/3068025568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cdbcdf0 0x56016cddf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.116+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/3068025568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cddf1f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.116+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/3068025568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cd94fb0 0x56016cd707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.116+0000 7f9be1a6a740 1 -- 172.21.15.98:0/3068025568 >> 172.21.15.98:0/3068025568 conn(0x56016cd725f0 msgr2=0x56016cd9dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:15.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.116+0000 7f9be1a6a740 1 -- 172.21.15.98:0/3068025568 shutdown_connections 2024-09-10T21:00:15.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.116+0000 7f9be1a6a740 1 -- 172.21.15.98:0/3068025568 wait complete. 2024-09-10T21:00:15.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.117+0000 7f9be1a6a740 1 -- start start 2024-09-10T21:00:15.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.117+0000 7f9be1a6a740 1 Processor -- start 2024-09-10T21:00:15.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9be1a6a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd94fb0 0x56016ce39d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9be1a6a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cd707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9be1a6a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cdbcdf0 0x56016cd70ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9be1a6a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56016cdced50 con 0x56016cd9d1b0 2024-09-10T21:00:15.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9be1a6a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x56016cd95620 con 0x56016cdbcdf0 2024-09-10T21:00:15.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9be1a6a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x56016cdcf9b0 con 0x56016cd94fb0 2024-09-10T21:00:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9be1a6a740 10 client.0 _ll_register_callbacks cb 0x7f9bd4003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:00:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cd707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cd707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41272/0 (socket says 172.21.15.98:41272) 2024-09-10T21:00:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bcffff640 1 -- 172.21.15.98:0/3841458565 learned_addr learned my addr 172.21.15.98:0/3841458565 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bdca37640 1 --2- 172.21.15.98:0/3841458565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd94fb0 0x56016ce39d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bdd238640 1 --2- 172.21.15.98:0/3841458565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cdbcdf0 0x56016cd70ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bcffff640 1 -- 172.21.15.98:0/3841458565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd94fb0 msgr2=0x56016ce39d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:00:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bcffff640 1 --2- 172.21.15.98:0/3841458565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd94fb0 0x56016ce39d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bcffff640 1 -- 172.21.15.98:0/3841458565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cdbcdf0 msgr2=0x56016cd70ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:00:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bcffff640 1 --2- 172.21.15.98:0/3841458565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cdbcdf0 0x56016cd70ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bcffff640 1 -- 172.21.15.98:0/3841458565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56016ce3c8a0 con 0x56016cd9d1b0 2024-09-10T21:00:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bcffff640 1 --2- 172.21.15.98:0/3841458565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cd707a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800d450 tx=0x7f9bc8003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34572c87bcfd304d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-10T21:00:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bb77fe640 1 -- 172.21.15.98:0/3841458565 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80049a0 con 0x56016cd9d1b0 2024-09-10T21:00:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bb77fe640 1 -- 172.21.15.98:0/3841458565 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f9bc8005030 con 0x56016cd9d1b0 2024-09-10T21:00:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9be1a6a740 1 -- 172.21.15.98:0/3841458565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc8004c30 con 0x56016cd9d1b0 2024-09-10T21:00:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9bb77fe640 1 -- 172.21.15.98:0/3841458565 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9bc8042050 con 0x56016cd9d1b0 2024-09-10T21:00:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.118+0000 7f9be1a6a740 10 client.5271 fetch_fsmap learned FSMap version 23 2024-09-10T21:00:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.119+0000 7f9be1a6a740 1 -- 172.21.15.98:0/3841458565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x56016ce3d460 con 0x56016cd9d1b0 2024-09-10T21:00:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.119+0000 7f9bb77fe640 1 -- 172.21.15.98:0/3841458565 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bc8004c30 con 0x56016cd9d1b0 2024-09-10T21:00:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.119+0000 7f9bb77fe640 1 -- 172.21.15.98:0/3841458565 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f9bc8045020 con 0x56016cd9d1b0 2024-09-10T21:00:15.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.119+0000 7f9be1a6a740 10 client.5271 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T21:00:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.119+0000 7f9be1a6a740 10 client.5271 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:00:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.119+0000 7f9be1a6a740 20 client.5271 populate_metadata read hostname 'smithi098' 2024-09-10T21:00:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.119+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/3841458565 >> [v2:172.21.15.190:6836/3087107175,v1:172.21.15.190:6837/3087107175] conn(0x56016ce40550 0x56016ce40930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.119+0000 7f9be1a6a740 4 client.5271 mds_command: new command op to 4834 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:00:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.119+0000 7f9be1a6a740 1 -- 172.21.15.98:0/3841458565 --> [v2:172.21.15.190:6836/3087107175,v1:172.21.15.190:6837/3087107175] -- command(tid 0: {"prefix": "status"}) -- 0x56016cce7220 con 0x56016ce40550 2024-09-10T21:00:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.119+0000 7f9bdca37640 1 --2- 172.21.15.98:0/3841458565 >> [v2:172.21.15.190:6836/3087107175,v1:172.21.15.190:6837/3087107175] conn(0x56016ce40550 0x56016ce40930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.119+0000 7f9bb77fe640 10 client.5271 ms_handle_connect on v2:172.21.15.190:6836/3087107175 2024-09-10T21:00:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.119+0000 7f9bdca37640 1 --2- 172.21.15.98:0/3841458565 >> [v2:172.21.15.190:6836/3087107175,v1:172.21.15.190:6837/3087107175] conn(0x56016ce40550 0x56016ce40930 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29e868a88abd287e server_cookie=105338e8ce2e00c0 in_seq=0 out_seq=0 2024-09-10T21:00:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9bb77fe640 1 -- 172.21.15.98:0/3841458565 <== mds.0 v2:172.21.15.190:6836/3087107175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56016cce7220 con 0x56016ce40550 2024-09-10T21:00:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9bb77fe640 10 client.5271 handle_command_reply: tid=0 2024-09-10T21:00:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9be1a6a740 1 client.5271 shutdown 2024-09-10T21:00:15.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9be1a6a740 1 -- 172.21.15.98:0/3841458565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 msgr2=0x56016cd707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/3841458565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cd707a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800d450 tx=0x7f9bc8003dd0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9be1a6a740 1 -- 172.21.15.98:0/3841458565 shutdown_connections 2024-09-10T21:00:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/3841458565 >> [v2:172.21.15.190:6836/3087107175,v1:172.21.15.190:6837/3087107175] conn(0x56016ce40550 0x56016ce40930 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/3841458565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56016cdbcdf0 0x56016cd70ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/3841458565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56016cd9d1b0 0x56016cd707a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9bb77fe640 0 client.5271 ms_handle_reset on v2:172.21.15.190:6836/3087107175 2024-09-10T21:00:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9be1a6a740 1 --2- 172.21.15.98:0/3841458565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56016cd94fb0 0x56016ce39d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9be1a6a740 1 -- 172.21.15.98:0/3841458565 >> 172.21.15.98:0/3841458565 conn(0x56016cd725f0 msgr2=0x56016cdbd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9be1a6a740 1 -- 172.21.15.98:0/3841458565 shutdown_connections 2024-09-10T21:00:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9be1a6a740 1 -- 172.21.15.98:0/3841458565 wait complete. 2024-09-10T21:00:15.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9be1a6a740 20 client.5271 trim_cache size 0 max 16384 2024-09-10T21:00:15.132 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-10T21:00:15.132 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "41041377-a016-49fd-be1f-323d0b834557", 2024-09-10T21:00:15.132 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-10T21:00:15.132 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 4834, 2024-09-10T21:00:15.132 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-10T21:00:15.132 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-10T21:00:15.132 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-10T21:00:15.132 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 6.1488823909999999, 2024-09-10T21:00:15.132 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 23, 2024-09-10T21:00:15.132 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 41, 2024-09-10T21:00:15.133 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 41, 2024-09-10T21:00:15.133 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 35.402737557999998 2024-09-10T21:00:15.133 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-10T21:00:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9bdec9b640 1 -- 172.21.15.98:0/868047884 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9bc4077f70 msgr2=0x7f9bc407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9bdec9b640 1 --2- 172.21.15.98:0/868047884 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9bc4077f70 0x7f9bc407a430 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4005fd0 tx=0x7f9bd4009660 comp rx=0 tx=0).stop 2024-09-10T21:00:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9bdec9b640 1 -- 172.21.15.98:0/868047884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd81f4ce0 msgr2=0x7f9bd80afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.120+0000 7f9bdec9b640 1 --2- 172.21.15.98:0/868047884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd81f4ce0 0x7f9bd80afe90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000c380 tx=0x7f9bd0006650 comp rx=0 tx=0).stop 2024-09-10T21:00:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.121+0000 7f9bdec9b640 1 -- 172.21.15.98:0/868047884 shutdown_connections 2024-09-10T21:00:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.121+0000 7f9bdec9b640 1 --2- 172.21.15.98:0/868047884 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9bc4077f70 0x7f9bc407a430 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.121+0000 7f9bdec9b640 1 --2- 172.21.15.98:0/868047884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bd81f4ce0 0x7f9bd80afe90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.121+0000 7f9bdec9b640 1 --2- 172.21.15.98:0/868047884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bd815af40 0x7f9bd81f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.121+0000 7f9bdec9b640 1 --2- 172.21.15.98:0/868047884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bc0004d40 0x7f9bd813e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.121+0000 7f9bdec9b640 1 -- 172.21.15.98:0/868047884 >> 172.21.15.98:0/868047884 conn(0x7f9bd80b1760 msgr2=0x7f9bd80cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:15.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.121+0000 7f9bdec9b640 1 -- 172.21.15.98:0/868047884 shutdown_connections 2024-09-10T21:00:15.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.121+0000 7f9bdec9b640 1 -- 172.21.15.98:0/868047884 wait complete. 2024-09-10T21:00:15.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:15.136 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%a /home/ubuntu/cephtest/mnt.0/testdir/testfile) 2024-09-10T21:00:15.152 INFO:teuthology.orchestra.run.smithi098.stdout:600 2024-09-10T21:00:15.153 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:00:15.153 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:00:15.234 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:00:15.280 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:00:15.280 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:00:15.280 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T21:00:15.280 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:00:15.296 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T21:00:15.296 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:00:15.352 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:00:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.489+0000 7f68ad16b640 1 -- 172.21.15.98:0/2759706040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68a815f230 msgr2=0x7f68a80b5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.489+0000 7f68ad16b640 1 --2- 172.21.15.98:0/2759706040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68a815f230 0x7f68a80b5950 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6898009830 tx=0x7f689802a530 comp rx=0 tx=0).stop 2024-09-10T21:00:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.490+0000 7f68ad16b640 1 -- 172.21.15.98:0/2759706040 shutdown_connections 2024-09-10T21:00:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.490+0000 7f68ad16b640 1 --2- 172.21.15.98:0/2759706040 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68a80d09c0 0x7f68a80b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.490+0000 7f68ad16b640 1 --2- 172.21.15.98:0/2759706040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68a815f230 0x7f68a80b5950 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.490+0000 7f68ad16b640 1 -- 172.21.15.98:0/2759706040 >> 172.21.15.98:0/2759706040 conn(0x7f68a8074150 msgr2=0x7f68a8074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.490+0000 7f68ad16b640 1 -- 172.21.15.98:0/2759706040 shutdown_connections 2024-09-10T21:00:15.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.490+0000 7f68ad16b640 1 -- 172.21.15.98:0/2759706040 wait complete. 2024-09-10T21:00:15.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.491+0000 7f68ad16b640 1 Processor -- start 2024-09-10T21:00:15.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.491+0000 7f68ad16b640 1 -- start start 2024-09-10T21:00:15.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68ad16b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68a80694d0 0x7f68a80cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68ad16b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68a80d09c0 0x7f68a8158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68ad16b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68a815f230 0x7f68a81f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68ad16b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f68a80b0ed0 con 0x7f68a80694d0 2024-09-10T21:00:15.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68ad16b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f68a80b0d50 con 0x7f68a815f230 2024-09-10T21:00:15.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68ad16b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f68a80b1050 con 0x7f68a80d09c0 2024-09-10T21:00:15.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68a6d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68a80694d0 0x7f68a80cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68a6d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68a80694d0 0x7f68a80cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41290/0 (socket says 172.21.15.98:41290) 2024-09-10T21:00:15.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68a6d9d640 1 -- 172.21.15.98:0/2065290939 learned_addr learned my addr 172.21.15.98:0/2065290939 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:15.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f689e59c640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68a80d09c0 0x7f68a8158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68a759e640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68a815f230 0x7f68a81f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68a6d9d640 1 -- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68a80d09c0 msgr2=0x7f68a8158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68a6d9d640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68a80d09c0 0x7f68a8158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68a6d9d640 1 -- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68a815f230 msgr2=0x7f68a81f54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68a6d9d640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68a815f230 0x7f68a81f54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68a6d9d640 1 -- 172.21.15.98:0/2065290939 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a81f8a60 con 0x7f68a80694d0 2024-09-10T21:00:15.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68a759e640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68a815f230 0x7f68a81f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:15.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.492+0000 7f68a6d9d640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68a80694d0 0x7f68a80cfb20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6898009950 tx=0x7f6898033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:15.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.493+0000 7f689ffff640 1 -- 172.21.15.98:0/2065290939 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898031d60 con 0x7f68a80694d0 2024-09-10T21:00:15.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.493+0000 7f689ffff640 1 -- 172.21.15.98:0/2065290939 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68980330d0 con 0x7f68a80694d0 2024-09-10T21:00:15.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.493+0000 7f68ad16b640 1 -- 172.21.15.98:0/2065290939 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a81f8d40 con 0x7f68a80694d0 2024-09-10T21:00:15.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.493+0000 7f689ffff640 1 -- 172.21.15.98:0/2065290939 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68980334e0 con 0x7f68a80694d0 2024-09-10T21:00:15.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.493+0000 7f68ad16b640 1 -- 172.21.15.98:0/2065290939 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a81f4cb0 con 0x7f68a80694d0 2024-09-10T21:00:15.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.494+0000 7f68ad16b640 1 -- 172.21.15.98:0/2065290939 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a8072560 con 0x7f68a80694d0 2024-09-10T21:00:15.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.494+0000 7f689ffff640 1 -- 172.21.15.98:0/2065290939 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f689804b030 con 0x7f68a80694d0 2024-09-10T21:00:15.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.495+0000 7f689ffff640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f688c077fd0 0x7f688c07a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.495+0000 7f689ffff640 1 -- 172.21.15.98:0/2065290939 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f68980baff0 con 0x7f68a80694d0 2024-09-10T21:00:15.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.495+0000 7f689e59c640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f688c077fd0 0x7f688c07a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.495+0000 7f689e59c640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f688c077fd0 0x7f688c07a490 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f68a81f24b0 tx=0x7f6890009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:15.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.499+0000 7f689ffff640 1 -- 172.21.15.98:0/2065290939 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f68a8072560 con 0x7f68a80694d0 2024-09-10T21:00:15.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.686+0000 7f68ad16b640 1 -- 172.21.15.98:0/2065290939 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f68a80b1560 con 0x7f68a80694d0 2024-09-10T21:00:15.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.686+0000 7f689ffff640 1 -- 172.21.15.98:0/2065290939 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v41) ==== 80+0+122 (secure 0 0 0) 0x7f689803f030 con 0x7f68a80694d0 2024-09-10T21:00:15.690 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98:6835/4202850803 2024-09-11T21:00:08.963614+0000 2024-09-10T21:00:15.690 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98:6832/4202850803 2024-09-11T21:00:08.963614+0000 2024-09-10T21:00:15.691 INFO:teuthology.orchestra.run.smithi098.stderr:listed 2 entries 2024-09-10T21:00:15.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.688+0000 7f68ad16b640 1 -- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f688c077fd0 msgr2=0x7f688c07a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.688+0000 7f68ad16b640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f688c077fd0 0x7f688c07a490 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f68a81f24b0 tx=0x7f6890009040 comp rx=0 tx=0).stop 2024-09-10T21:00:15.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.688+0000 7f68ad16b640 1 -- 172.21.15.98:0/2065290939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68a80694d0 msgr2=0x7f68a80cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.688+0000 7f68ad16b640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68a80694d0 0x7f68a80cfb20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6898009950 tx=0x7f6898033ce0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.688+0000 7f68ad16b640 1 -- 172.21.15.98:0/2065290939 shutdown_connections 2024-09-10T21:00:15.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.688+0000 7f68ad16b640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f688c077fd0 0x7f688c07a490 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.688+0000 7f68ad16b640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68a815f230 0x7f68a81f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.688+0000 7f68ad16b640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68a80d09c0 0x7f68a8158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.688+0000 7f68ad16b640 1 --2- 172.21.15.98:0/2065290939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68a80694d0 0x7f68a80cfb20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.688+0000 7f68ad16b640 1 -- 172.21.15.98:0/2065290939 >> 172.21.15.98:0/2065290939 conn(0x7f68a8074150 msgr2=0x7f68a81e84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:15.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.688+0000 7f68ad16b640 1 -- 172.21.15.98:0/2065290939 shutdown_connections 2024-09-10T21:00:15.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.688+0000 7f68ad16b640 1 -- 172.21.15.98:0/2065290939 wait complete. 2024-09-10T21:00:15.703 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:00:15.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:15.703 DEBUG:teuthology.orchestra.run.smithi098:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:00:15.731 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T21:00:15.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:15.732 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:00:15.787 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:00:15.788 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T21:00:15.788 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:00:15.808 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=154 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:00:15.808 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:00:15.808 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T21:00:15.808 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:00:15.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056024,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2547 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:00:15.843 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:00:15.843 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T21:00:15.843 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:00:15.866 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=154 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:00:15.866 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:00:15.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.980+0000 7f2d54e62640 1 -- 172.21.15.98:0/1182611614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d3c003ad0 msgr2=0x7f2d3c003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.980+0000 7f2d54e62640 1 --2- 172.21.15.98:0/1182611614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d3c003ad0 0x7f2d3c003ef0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2d40006f50 tx=0x7f2d3c005830 comp rx=0 tx=0).stop 2024-09-10T21:00:15.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.980+0000 7f2d4ed9d640 1 -- 172.21.15.98:0/1182611614 reap_dead start 2024-09-10T21:00:15.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.980+0000 7f2d54e62640 1 -- 172.21.15.98:0/1182611614 shutdown_connections 2024-09-10T21:00:15.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.980+0000 7f2d54e62640 1 -- 172.21.15.98:0/1182611614 >> 172.21.15.98:0/1182611614 conn(0x7f2d500d3f70 msgr2=0x7f2d500d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:15.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.980+0000 7f2d54e62640 1 -- 172.21.15.98:0/1182611614 shutdown_connections 2024-09-10T21:00:15.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.980+0000 7f2d54e62640 1 -- 172.21.15.98:0/1182611614 wait complete. 2024-09-10T21:00:15.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.982+0000 7f2d54e62640 1 Processor -- start 2024-09-10T21:00:15.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.982+0000 7f2d54e62640 1 -- start start 2024-09-10T21:00:15.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.982+0000 7f2d54e62640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d3c003ad0 0x7f2d500cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.982+0000 7f2d54e62640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d500a14e0 0x7f2d500cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.982+0000 7f2d54e62640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d50156d80 0x7f2d501571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.982+0000 7f2d54e62640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2d50166f70 con 0x7f2d50156d80 2024-09-10T21:00:15.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.982+0000 7f2d54e62640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2d50166df0 con 0x7f2d500a14e0 2024-09-10T21:00:15.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.982+0000 7f2d54e62640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2d501670f0 con 0x7f2d3c003ad0 2024-09-10T21:00:15.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.982+0000 7f2d4ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d50156d80 0x7f2d501571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.982+0000 7f2d4ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d50156d80 0x7f2d501571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41308/0 (socket says 172.21.15.98:41308) 2024-09-10T21:00:15.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d4ed9d640 1 -- 172.21.15.98:0/154629663 learned_addr learned my addr 172.21.15.98:0/154629663 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:15.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d4e59c640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d3c003ad0 0x7f2d500cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d47fff640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d500a14e0 0x7f2d500cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d4ed9d640 1 -- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d3c003ad0 msgr2=0x7f2d500cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d4ed9d640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d3c003ad0 0x7f2d500cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d4ed9d640 1 -- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d500a14e0 msgr2=0x7f2d500cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:15.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d4ed9d640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d500a14e0 0x7f2d500cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:15.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d4ed9d640 1 -- 172.21.15.98:0/154629663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d501f4da0 con 0x7f2d50156d80 2024-09-10T21:00:15.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d47fff640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d500a14e0 0x7f2d500cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:00:15.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d4e59c640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d3c003ad0 0x7f2d500cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:00:15.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d4ed9d640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d50156d80 0x7f2d501571e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2d48005fd0 tx=0x7f2d4800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:15.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d46ffd640 1 -- 172.21.15.98:0/154629663 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48004b20 con 0x7f2d50156d80 2024-09-10T21:00:15.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d46ffd640 1 -- 172.21.15.98:0/154629663 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d48004cc0 con 0x7f2d50156d80 2024-09-10T21:00:15.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d54e62640 1 -- 172.21.15.98:0/154629663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d501f5060 con 0x7f2d50156d80 2024-09-10T21:00:15.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d46ffd640 1 -- 172.21.15.98:0/154629663 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48003bd0 con 0x7f2d50156d80 2024-09-10T21:00:15.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.983+0000 7f2d54e62640 1 -- 172.21.15.98:0/154629663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d501f5620 con 0x7f2d50156d80 2024-09-10T21:00:15.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.984+0000 7f2d54e62640 1 -- 172.21.15.98:0/154629663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d500d24e0 con 0x7f2d50156d80 2024-09-10T21:00:15.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.985+0000 7f2d46ffd640 1 -- 172.21.15.98:0/154629663 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2d48018070 con 0x7f2d50156d80 2024-09-10T21:00:15.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.985+0000 7f2d46ffd640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2d30077f70 0x7f2d3007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:15.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.985+0000 7f2d46ffd640 1 -- 172.21.15.98:0/154629663 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f2d4809a3c0 con 0x7f2d50156d80 2024-09-10T21:00:15.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.985+0000 7f2d4e59c640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2d30077f70 0x7f2d3007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:15.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.986+0000 7f2d4e59c640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2d30077f70 0x7f2d3007a430 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2d40033000 tx=0x7f2d5008e5e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:15.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:15.989+0000 7f2d46ffd640 1 -- 172.21.15.98:0/154629663 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2d48062d70 con 0x7f2d50156d80 2024-09-10T21:00:16.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.196+0000 7f2d54e62640 1 -- 172.21.15.98:0/154629663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d500b72f0 con 0x7f2d50156d80 2024-09-10T21:00:16.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.197+0000 7f2d46ffd640 1 -- 172.21.15.98:0/154629663 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5231 (secure 0 0 0) 0x7f2d48062670 con 0x7f2d50156d80 2024-09-10T21:00:16.201 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:16.201 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":23,"btime":"2024-09-10T21:00:12:974795+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/1700293059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":1700293059},{"type":"v1","addr":"172.21.15.98:6834","nonce":1700293059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4048376378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4048376378},{"type":"v1","addr":"172.21.15.190:6835","nonce":4048376378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/2245019178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2245019178},{"type":"v1","addr":"172.21.15.98:6835","nonce":2245019178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:59:46.344123+0000","modified":"2024-09-10T21:00:12.974787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4834},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4834":{"gid":4834,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.190:6837/3087107175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3087107175},{"type":"v1","addr":"172.21.15.190:6837","nonce":3087107175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4834,"qdb_cluster":[4834]},"id":2}]} 2024-09-10T21:00:16.202 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 23 2024-09-10T21:00:16.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.199+0000 7f2d54e62640 1 -- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2d30077f70 msgr2=0x7f2d3007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.199+0000 7f2d54e62640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2d30077f70 0x7f2d3007a430 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2d40033000 tx=0x7f2d5008e5e0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.199+0000 7f2d54e62640 1 -- 172.21.15.98:0/154629663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d50156d80 msgr2=0x7f2d501571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.199+0000 7f2d54e62640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d50156d80 0x7f2d501571e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2d48005fd0 tx=0x7f2d4800f3c0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.199+0000 7f2d54e62640 1 -- 172.21.15.98:0/154629663 shutdown_connections 2024-09-10T21:00:16.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.199+0000 7f2d54e62640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2d30077f70 0x7f2d3007a430 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.199+0000 7f2d54e62640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d50156d80 0x7f2d501571e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.199+0000 7f2d54e62640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d500a14e0 0x7f2d500cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.199+0000 7f2d54e62640 1 --2- 172.21.15.98:0/154629663 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d3c003ad0 0x7f2d500cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.199+0000 7f2d54e62640 1 -- 172.21.15.98:0/154629663 >> 172.21.15.98:0/154629663 conn(0x7f2d500d3f70 msgr2=0x7f2d5015d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:16.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.199+0000 7f2d54e62640 1 -- 172.21.15.98:0/154629663 shutdown_connections 2024-09-10T21:00:16.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.199+0000 7f2d54e62640 1 -- 172.21.15.98:0/154629663 wait complete. 2024-09-10T21:00:16.215 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:00:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.312+0000 7f2b25633640 1 -- 172.21.15.98:0/730463836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b200b02a0 msgr2=0x7f2b20070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.312+0000 7f2b25633640 1 --2- 172.21.15.98:0/730463836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b200b02a0 0x7f2b20070060 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2b10005fd0 tx=0x7f2b1002a8a0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.313+0000 7f2b25633640 1 -- 172.21.15.98:0/730463836 shutdown_connections 2024-09-10T21:00:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.313+0000 7f2b25633640 1 --2- 172.21.15.98:0/730463836 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b20161880 0x7f2b200b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.313+0000 7f2b25633640 1 --2- 172.21.15.98:0/730463836 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b200b02a0 0x7f2b20070060 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.313+0000 7f2b25633640 1 -- 172.21.15.98:0/730463836 >> 172.21.15.98:0/730463836 conn(0x7f2b200b2770 msgr2=0x7f2b200b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.313+0000 7f2b25633640 1 -- 172.21.15.98:0/730463836 shutdown_connections 2024-09-10T21:00:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.313+0000 7f2b25633640 1 -- 172.21.15.98:0/730463836 wait complete. 2024-09-10T21:00:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.314+0000 7f2b25633640 1 Processor -- start 2024-09-10T21:00:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.314+0000 7f2b25633640 1 -- start start 2024-09-10T21:00:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b25633640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b200b02a0 0x7f2b20154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b25633640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b200b4580 0x7f2b20150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b25633640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b20161880 0x7f2b20151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b25633640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2006cbd0 con 0x7f2b200b02a0 2024-09-10T21:00:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b25633640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2006ca50 con 0x7f2b20161880 2024-09-10T21:00:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b25633640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2006cd50 con 0x7f2b200b4580 2024-09-10T21:00:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b1effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b200b02a0 0x7f2b20154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b1effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b200b02a0 0x7f2b20154130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41312/0 (socket says 172.21.15.98:41312) 2024-09-10T21:00:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b1effd640 1 -- 172.21.15.98:0/2723402063 learned_addr learned my addr 172.21.15.98:0/2723402063 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b1e7fc640 1 --2- 172.21.15.98:0/2723402063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b200b4580 0x7f2b20150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b1f7fe640 1 --2- 172.21.15.98:0/2723402063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b20161880 0x7f2b20151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b1effd640 1 -- 172.21.15.98:0/2723402063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b200b4580 msgr2=0x7f2b20150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b1effd640 1 --2- 172.21.15.98:0/2723402063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b200b4580 0x7f2b20150ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b1effd640 1 -- 172.21.15.98:0/2723402063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b20161880 msgr2=0x7f2b20151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b1effd640 1 --2- 172.21.15.98:0/2723402063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b20161880 0x7f2b20151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b1effd640 1 -- 172.21.15.98:0/2723402063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b20155a40 con 0x7f2b200b02a0 2024-09-10T21:00:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b1effd640 1 --2- 172.21.15.98:0/2723402063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b200b02a0 0x7f2b20154130 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2b10006010 tx=0x7f2b10003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b177fe640 1 -- 172.21.15.98:0/2723402063 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10004450 con 0x7f2b200b02a0 2024-09-10T21:00:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b177fe640 1 -- 172.21.15.98:0/2723402063 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b10004610 con 0x7f2b200b02a0 2024-09-10T21:00:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.315+0000 7f2b25633640 1 -- 172.21.15.98:0/2723402063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b20154c80 con 0x7f2b200b02a0 2024-09-10T21:00:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.316+0000 7f2b177fe640 1 -- 172.21.15.98:0/2723402063 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10003e10 con 0x7f2b200b02a0 2024-09-10T21:00:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.316+0000 7f2b25633640 1 -- 172.21.15.98:0/2723402063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b20155240 con 0x7f2b200b02a0 2024-09-10T21:00:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.317+0000 7f2b25633640 1 -- 172.21.15.98:0/2723402063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b2006d1d0 con 0x7f2b200b02a0 2024-09-10T21:00:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.317+0000 7f2b177fe640 1 -- 172.21.15.98:0/2723402063 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2b10030070 con 0x7f2b200b02a0 2024-09-10T21:00:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.317+0000 7f2b177fe640 1 --2- 172.21.15.98:0/2723402063 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2b00077f70 0x7f2b0007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.318+0000 7f2b177fe640 1 -- 172.21.15.98:0/2723402063 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f2b1002e020 con 0x7f2b200b02a0 2024-09-10T21:00:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.318+0000 7f2b1e7fc640 1 --2- 172.21.15.98:0/2723402063 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2b00077f70 0x7f2b0007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.318+0000 7f2b1e7fc640 1 --2- 172.21.15.98:0/2723402063 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2b00077f70 0x7f2b0007a430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2b20071640 tx=0x7f2b08005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:16.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.322+0000 7f2b177fe640 1 -- 172.21.15.98:0/2723402063 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2b100835c0 con 0x7f2b200b02a0 2024-09-10T21:00:16.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.530+0000 7f2b25633640 1 -- 172.21.15.98:0/2723402063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b20161580 con 0x7f2b200b02a0 2024-09-10T21:00:16.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.531+0000 7f2b177fe640 1 -- 172.21.15.98:0/2723402063 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5231 (secure 0 0 0) 0x7f2b10087ff0 con 0x7f2b200b02a0 2024-09-10T21:00:16.535 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:16.535 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":23,"btime":"2024-09-10T21:00:12:974795+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/1700293059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":1700293059},{"type":"v1","addr":"172.21.15.98:6834","nonce":1700293059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4048376378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4048376378},{"type":"v1","addr":"172.21.15.190:6835","nonce":4048376378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/2245019178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2245019178},{"type":"v1","addr":"172.21.15.98:6835","nonce":2245019178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:59:46.344123+0000","modified":"2024-09-10T21:00:12.974787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4834},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4834":{"gid":4834,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.190:6837/3087107175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3087107175},{"type":"v1","addr":"172.21.15.190:6837","nonce":3087107175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4834,"qdb_cluster":[4834]},"id":2}]} 2024-09-10T21:00:16.536 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 23 2024-09-10T21:00:16.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.533+0000 7f2b25633640 1 -- 172.21.15.98:0/2723402063 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2b00077f70 msgr2=0x7f2b0007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.533+0000 7f2b25633640 1 --2- 172.21.15.98:0/2723402063 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2b00077f70 0x7f2b0007a430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2b20071640 tx=0x7f2b08005f90 comp rx=0 tx=0).stop 2024-09-10T21:00:16.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.533+0000 7f2b25633640 1 -- 172.21.15.98:0/2723402063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b200b02a0 msgr2=0x7f2b20154130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.533+0000 7f2b25633640 1 --2- 172.21.15.98:0/2723402063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b200b02a0 0x7f2b20154130 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2b10006010 tx=0x7f2b10003c20 comp rx=0 tx=0).stop 2024-09-10T21:00:16.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.533+0000 7f2b25633640 1 -- 172.21.15.98:0/2723402063 shutdown_connections 2024-09-10T21:00:16.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.533+0000 7f2b25633640 1 --2- 172.21.15.98:0/2723402063 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2b00077f70 0x7f2b0007a430 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.533+0000 7f2b25633640 1 --2- 172.21.15.98:0/2723402063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b20161880 0x7f2b20151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.533+0000 7f2b25633640 1 --2- 172.21.15.98:0/2723402063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b200b4580 0x7f2b20150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.533+0000 7f2b25633640 1 --2- 172.21.15.98:0/2723402063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b200b02a0 0x7f2b20154130 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.533+0000 7f2b25633640 1 -- 172.21.15.98:0/2723402063 >> 172.21.15.98:0/2723402063 conn(0x7f2b200b2770 msgr2=0x7f2b200b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:16.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.533+0000 7f2b25633640 1 -- 172.21.15.98:0/2723402063 shutdown_connections 2024-09-10T21:00:16.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.533+0000 7f2b25633640 1 -- 172.21.15.98:0/2723402063 wait complete. 2024-09-10T21:00:16.548 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:00:16.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.643+0000 7f47a8a72640 1 -- 172.21.15.98:0/3927431985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a415e150 msgr2=0x7f47a415e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.643+0000 7f47a8a72640 1 --2- 172.21.15.98:0/3927431985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a415e150 0x7f47a415e530 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f47a40af670 tx=0x7f479c02a8b0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.643+0000 7f47a8a72640 1 -- 172.21.15.98:0/3927431985 shutdown_connections 2024-09-10T21:00:16.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.643+0000 7f47a8a72640 1 --2- 172.21.15.98:0/3927431985 >> v2:172.21.15.190:3300/0 conn(0x7f47a406e2e0 0x7f47a40b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.643+0000 7f47a8a72640 1 --2- 172.21.15.98:0/3927431985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a415e150 0x7f47a415e530 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.643+0000 7f47a8a72640 1 -- 172.21.15.98:0/3927431985 >> 172.21.15.98:0/3927431985 conn(0x7f47a40b1760 msgr2=0x7f47a40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:16.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.643+0000 7f47a8a72640 1 -- 172.21.15.98:0/3927431985 shutdown_connections 2024-09-10T21:00:16.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.643+0000 7f47a8a72640 1 -- 172.21.15.98:0/3927431985 wait complete. 2024-09-10T21:00:16.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.645+0000 7f47a8a72640 1 Processor -- start 2024-09-10T21:00:16.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.645+0000 7f47a8a72640 1 -- start start 2024-09-10T21:00:16.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.645+0000 7f47a8a72640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a406e2e0 0x7f47a41f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:16.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.645+0000 7f47a8a72640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a40725d0 0x7f47a4132010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:16.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.645+0000 7f47a8a72640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47a415e150 0x7f47a40cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:16.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.645+0000 7f47a8a72640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f47a40cc570 con 0x7f47a406e2e0 2024-09-10T21:00:16.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.645+0000 7f47a8a72640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f47a40cf3e0 con 0x7f47a415e150 2024-09-10T21:00:16.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.645+0000 7f47a8a72640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f47a40cc6f0 con 0x7f47a40725d0 2024-09-10T21:00:16.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f47a259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a406e2e0 0x7f47a41f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:16.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f47a259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a406e2e0 0x7f47a41f6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41326/0 (socket says 172.21.15.98:41326) 2024-09-10T21:00:16.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f47a259c640 1 -- 172.21.15.98:0/2963873895 learned_addr learned my addr 172.21.15.98:0/2963873895 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:16.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f47a2d9d640 1 --2- 172.21.15.98:0/2963873895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47a415e150 0x7f47a40cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:16.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f47a1d9b640 1 --2- 172.21.15.98:0/2963873895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a40725d0 0x7f47a4132010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:16.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f47a259c640 1 -- 172.21.15.98:0/2963873895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a40725d0 msgr2=0x7f47a4132010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f47a259c640 1 --2- 172.21.15.98:0/2963873895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a40725d0 0x7f47a4132010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f47a259c640 1 -- 172.21.15.98:0/2963873895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47a415e150 msgr2=0x7f47a40cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f47a259c640 1 --2- 172.21.15.98:0/2963873895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47a415e150 0x7f47a40cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f47a259c640 1 -- 172.21.15.98:0/2963873895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a41f2ee0 con 0x7f47a406e2e0 2024-09-10T21:00:16.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f47a259c640 1 --2- 172.21.15.98:0/2963873895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a406e2e0 0x7f47a41f6920 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f479c009ca0 tx=0x7f479c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:16.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f479affd640 1 -- 172.21.15.98:0/2963873895 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c031da0 con 0x7f47a406e2e0 2024-09-10T21:00:16.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f479affd640 1 -- 172.21.15.98:0/2963873895 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f479c003e10 con 0x7f47a406e2e0 2024-09-10T21:00:16.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f479affd640 1 -- 172.21.15.98:0/2963873895 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c033310 con 0x7f47a406e2e0 2024-09-10T21:00:16.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f47a8a72640 1 -- 172.21.15.98:0/2963873895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a41f31c0 con 0x7f47a406e2e0 2024-09-10T21:00:16.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.646+0000 7f47a8a72640 1 -- 172.21.15.98:0/2963873895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a41f3780 con 0x7f47a406e2e0 2024-09-10T21:00:16.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.647+0000 7f47a8a72640 1 -- 172.21.15.98:0/2963873895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a415d940 con 0x7f47a406e2e0 2024-09-10T21:00:16.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.648+0000 7f479affd640 1 -- 172.21.15.98:0/2963873895 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f479c0334b0 con 0x7f47a406e2e0 2024-09-10T21:00:16.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.648+0000 7f479affd640 1 --2- 172.21.15.98:0/2963873895 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4784077f70 0x7f478407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:16.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.648+0000 7f479affd640 1 -- 172.21.15.98:0/2963873895 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f479c0bb4f0 con 0x7f47a406e2e0 2024-09-10T21:00:16.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.648+0000 7f47a1d9b640 1 --2- 172.21.15.98:0/2963873895 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4784077f70 0x7f478407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:16.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.649+0000 7f47a1d9b640 1 --2- 172.21.15.98:0/2963873895 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4784077f70 0x7f478407a430 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f478c005e10 tx=0x7f478c005da0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:16.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.652+0000 7f479affd640 1 -- 172.21.15.98:0/2963873895 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f479c03f030 con 0x7f47a406e2e0 2024-09-10T21:00:16.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.828+0000 7f47a8a72640 1 -- 172.21.15.98:0/2963873895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47a40b2e70 con 0x7f47a406e2e0 2024-09-10T21:00:16.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.829+0000 7f479affd640 1 -- 172.21.15.98:0/2963873895 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17143 (secure 0 0 0) 0x7f479c0337d0 con 0x7f47a406e2e0 2024-09-10T21:00:16.833 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:16.833 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":41,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T21:00:08.963642+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:59:44.162419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T20:59:45.314201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6835/4202850803":"2024-09-11T21:00:08.963614+0000","172.21.15.98:6832/4202850803":"2024-09-11T21:00:08.963614+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:00:16.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.831+0000 7f47a8a72640 1 -- 172.21.15.98:0/2963873895 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4784077f70 msgr2=0x7f478407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.831+0000 7f47a8a72640 1 --2- 172.21.15.98:0/2963873895 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4784077f70 0x7f478407a430 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f478c005e10 tx=0x7f478c005da0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.831+0000 7f47a8a72640 1 -- 172.21.15.98:0/2963873895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a406e2e0 msgr2=0x7f47a41f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.831+0000 7f47a8a72640 1 --2- 172.21.15.98:0/2963873895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a406e2e0 0x7f47a41f6920 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f479c009ca0 tx=0x7f479c003c20 comp rx=0 tx=0).stop 2024-09-10T21:00:16.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.831+0000 7f47a8a72640 1 -- 172.21.15.98:0/2963873895 shutdown_connections 2024-09-10T21:00:16.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.831+0000 7f47a8a72640 1 --2- 172.21.15.98:0/2963873895 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f4784077f70 0x7f478407a430 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.831+0000 7f47a8a72640 1 --2- 172.21.15.98:0/2963873895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47a415e150 0x7f47a40cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.831+0000 7f47a8a72640 1 --2- 172.21.15.98:0/2963873895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47a40725d0 0x7f47a4132010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.831+0000 7f47a8a72640 1 --2- 172.21.15.98:0/2963873895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a406e2e0 0x7f47a41f6920 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.831+0000 7f47a8a72640 1 -- 172.21.15.98:0/2963873895 >> 172.21.15.98:0/2963873895 conn(0x7f47a40b1760 msgr2=0x7f47a406f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:16.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.831+0000 7f47a8a72640 1 -- 172.21.15.98:0/2963873895 shutdown_connections 2024-09-10T21:00:16.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.831+0000 7f47a8a72640 1 -- 172.21.15.98:0/2963873895 wait complete. 2024-09-10T21:00:16.846 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:00:16.847 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:00:16.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.943+0000 7f440b3bc640 1 -- 172.21.15.98:0/2358229623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4404072680 msgr2=0x7f44040ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.943+0000 7f440b3bc640 1 --2- 172.21.15.98:0/2358229623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4404072680 0x7f44040ceb00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4400009830 tx=0x7f440002a480 comp rx=0 tx=0).stop 2024-09-10T21:00:16.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.943+0000 7f440b3bc640 1 -- 172.21.15.98:0/2358229623 shutdown_connections 2024-09-10T21:00:16.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.943+0000 7f440b3bc640 1 --2- 172.21.15.98:0/2358229623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4404072ad0 0x7f44040cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.943+0000 7f440b3bc640 1 --2- 172.21.15.98:0/2358229623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4404072680 0x7f44040ceb00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.943+0000 7f440b3bc640 1 -- 172.21.15.98:0/2358229623 >> 172.21.15.98:0/2358229623 conn(0x7f44040b1760 msgr2=0x7f44040b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:16.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.943+0000 7f440b3bc640 1 -- 172.21.15.98:0/2358229623 shutdown_connections 2024-09-10T21:00:16.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.943+0000 7f440b3bc640 1 -- 172.21.15.98:0/2358229623 wait complete. 2024-09-10T21:00:16.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.944+0000 7f440b3bc640 1 Processor -- start 2024-09-10T21:00:16.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.945+0000 7f440b3bc640 1 -- start start 2024-09-10T21:00:16.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.945+0000 7f440b3bc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4404072680 0x7f44040c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:16.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.945+0000 7f440b3bc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4404072ad0 0x7f44041f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:16.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.945+0000 7f440b3bc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4404144780 0x7f44040c5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:16.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.945+0000 7f440b3bc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f440415c9f0 con 0x7f4404144780 2024-09-10T21:00:16.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.945+0000 7f440b3bc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f440415c870 con 0x7f4404072680 2024-09-10T21:00:16.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.945+0000 7f440b3bc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f440415cb70 con 0x7f4404072ad0 2024-09-10T21:00:16.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.945+0000 7f4409959640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4404144780 0x7f44040c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:16.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.945+0000 7f4409959640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4404144780 0x7f44040c5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41336/0 (socket says 172.21.15.98:41336) 2024-09-10T21:00:16.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.945+0000 7f4409959640 1 -- 172.21.15.98:0/2928559624 learned_addr learned my addr 172.21.15.98:0/2928559624 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:16.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.946+0000 7f4408957640 1 --2- 172.21.15.98:0/2928559624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4404072ad0 0x7f44041f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:16.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.946+0000 7f4409158640 1 --2- 172.21.15.98:0/2928559624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4404072680 0x7f44040c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:16.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.946+0000 7f4409959640 1 -- 172.21.15.98:0/2928559624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4404072ad0 msgr2=0x7f44041f7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.946+0000 7f4409959640 1 --2- 172.21.15.98:0/2928559624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4404072ad0 0x7f44041f7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.946+0000 7f4409959640 1 -- 172.21.15.98:0/2928559624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4404072680 msgr2=0x7f44040c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.946+0000 7f4409959640 1 --2- 172.21.15.98:0/2928559624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4404072680 0x7f44040c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.946+0000 7f4409959640 1 -- 172.21.15.98:0/2928559624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44040c4f10 con 0x7f4404144780 2024-09-10T21:00:16.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.946+0000 7f4409959640 1 --2- 172.21.15.98:0/2928559624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4404144780 0x7f44040c5470 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00eaa0 tx=0x7f43fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:16.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.946+0000 7f43f5ffb640 1 -- 172.21.15.98:0/2928559624 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc00cdb0 con 0x7f4404144780 2024-09-10T21:00:16.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.946+0000 7f43f5ffb640 1 -- 172.21.15.98:0/2928559624 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43fc004c10 con 0x7f4404144780 2024-09-10T21:00:16.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.946+0000 7f440b3bc640 1 -- 172.21.15.98:0/2928559624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44040ae4f0 con 0x7f4404144780 2024-09-10T21:00:16.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.946+0000 7f43f5ffb640 1 -- 172.21.15.98:0/2928559624 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc005020 con 0x7f4404144780 2024-09-10T21:00:16.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.946+0000 7f440b3bc640 1 -- 172.21.15.98:0/2928559624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44040aeab0 con 0x7f4404144780 2024-09-10T21:00:16.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.947+0000 7f440b3bc640 1 -- 172.21.15.98:0/2928559624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f440415d020 con 0x7f4404144780 2024-09-10T21:00:16.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.947+0000 7f43f5ffb640 1 -- 172.21.15.98:0/2928559624 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f43fc00f040 con 0x7f4404144780 2024-09-10T21:00:16.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.948+0000 7f43f5ffb640 1 --2- 172.21.15.98:0/2928559624 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f43e8077f70 0x7f43e807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:16.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.948+0000 7f43f5ffb640 1 -- 172.21.15.98:0/2928559624 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f43fc09a5b0 con 0x7f4404144780 2024-09-10T21:00:16.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.948+0000 7f4409158640 1 --2- 172.21.15.98:0/2928559624 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f43e8077f70 0x7f43e807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:16.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.949+0000 7f4409158640 1 --2- 172.21.15.98:0/2928559624 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f43e8077f70 0x7f43e807a430 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f44040af6a0 tx=0x7f43f8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:16.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:16.952+0000 7f43f5ffb640 1 -- 172.21.15.98:0/2928559624 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f43fc016030 con 0x7f4404144780 2024-09-10T21:00:17.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.159+0000 7f440b3bc640 1 -- 172.21.15.98:0/2928559624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44040d0220 con 0x7f4404144780 2024-09-10T21:00:17.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.160+0000 7f43f5ffb640 1 -- 172.21.15.98:0/2928559624 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5231 (secure 0 0 0) 0x7f43fc062ee0 con 0x7f4404144780 2024-09-10T21:00:17.164 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:17.164 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":23,"btime":"2024-09-10T21:00:12:974795+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/1700293059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":1700293059},{"type":"v1","addr":"172.21.15.98:6834","nonce":1700293059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4048376378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4048376378},{"type":"v1","addr":"172.21.15.190:6835","nonce":4048376378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/2245019178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2245019178},{"type":"v1","addr":"172.21.15.98:6835","nonce":2245019178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:59:46.344123+0000","modified":"2024-09-10T21:00:12.974787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4834},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4834":{"gid":4834,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.190:6837/3087107175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3087107175},{"type":"v1","addr":"172.21.15.190:6837","nonce":3087107175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4834,"qdb_cluster":[4834]},"id":2}]} 2024-09-10T21:00:17.165 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 23 2024-09-10T21:00:17.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.162+0000 7f440b3bc640 1 -- 172.21.15.98:0/2928559624 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f43e8077f70 msgr2=0x7f43e807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.162+0000 7f440b3bc640 1 --2- 172.21.15.98:0/2928559624 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f43e8077f70 0x7f43e807a430 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f44040af6a0 tx=0x7f43f8009040 comp rx=0 tx=0).stop 2024-09-10T21:00:17.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.162+0000 7f440b3bc640 1 -- 172.21.15.98:0/2928559624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4404144780 msgr2=0x7f44040c5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.162+0000 7f440b3bc640 1 --2- 172.21.15.98:0/2928559624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4404144780 0x7f44040c5470 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00eaa0 tx=0x7f43fc00ef60 comp rx=0 tx=0).stop 2024-09-10T21:00:17.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.162+0000 7f440b3bc640 1 -- 172.21.15.98:0/2928559624 shutdown_connections 2024-09-10T21:00:17.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.162+0000 7f440b3bc640 1 --2- 172.21.15.98:0/2928559624 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f43e8077f70 0x7f43e807a430 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.162+0000 7f440b3bc640 1 --2- 172.21.15.98:0/2928559624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4404144780 0x7f44040c5470 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.162+0000 7f440b3bc640 1 --2- 172.21.15.98:0/2928559624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4404072ad0 0x7f44041f7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.162+0000 7f440b3bc640 1 --2- 172.21.15.98:0/2928559624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4404072680 0x7f44040c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.162+0000 7f440b3bc640 1 -- 172.21.15.98:0/2928559624 >> 172.21.15.98:0/2928559624 conn(0x7f44040b1760 msgr2=0x7f44040cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:17.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.162+0000 7f440b3bc640 1 -- 172.21.15.98:0/2928559624 shutdown_connections 2024-09-10T21:00:17.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.162+0000 7f440b3bc640 1 -- 172.21.15.98:0/2928559624 wait complete. 2024-09-10T21:00:17.177 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:00:17.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.273+0000 7f8eec052640 1 -- 172.21.15.98:0/3808343444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee4144780 msgr2=0x7f8ee40b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.273+0000 7f8eec052640 1 --2- 172.21.15.98:0/3808343444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee4144780 0x7f8ee40b3da0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009830 tx=0x7f8ed802a480 comp rx=0 tx=0).stop 2024-09-10T21:00:17.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.273+0000 7f8eec052640 1 -- 172.21.15.98:0/3808343444 shutdown_connections 2024-09-10T21:00:17.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.274+0000 7f8eec052640 1 --2- 172.21.15.98:0/3808343444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ee4072680 0x7f8ee4072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.274+0000 7f8eec052640 1 --2- 172.21.15.98:0/3808343444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee4144780 0x7f8ee40b3da0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.274+0000 7f8eec052640 1 -- 172.21.15.98:0/3808343444 >> 172.21.15.98:0/3808343444 conn(0x7f8ee40b1760 msgr2=0x7f8ee40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:17.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.274+0000 7f8eec052640 1 -- 172.21.15.98:0/3808343444 shutdown_connections 2024-09-10T21:00:17.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.274+0000 7f8eec052640 1 -- 172.21.15.98:0/3808343444 wait complete. 2024-09-10T21:00:17.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.275+0000 7f8eec052640 1 Processor -- start 2024-09-10T21:00:17.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.275+0000 7f8eec052640 1 -- start start 2024-09-10T21:00:17.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.275+0000 7f8eec052640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ee4072680 0x7f8ee40c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:17.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.275+0000 7f8eec052640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee40ceb00 0x7f8ee4159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:17.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.275+0000 7f8eec052640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ee4144780 0x7f8ee40ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.275+0000 7f8eec052640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee415d070 con 0x7f8ee40ceb00 2024-09-10T21:00:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.275+0000 7f8eec052640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee415cef0 con 0x7f8ee4072680 2024-09-10T21:00:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.275+0000 7f8eec052640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8ee40cef70 con 0x7f8ee4144780 2024-09-10T21:00:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8ee95ed640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee40ceb00 0x7f8ee4159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8ee95ed640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee40ceb00 0x7f8ee4159ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41342/0 (socket says 172.21.15.98:41342) 2024-09-10T21:00:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8ee95ed640 1 -- 172.21.15.98:0/3267937189 learned_addr learned my addr 172.21.15.98:0/3267937189 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8ee9dee640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ee4072680 0x7f8ee40c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8eea5ef640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ee4144780 0x7f8ee40ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8ee95ed640 1 -- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ee4144780 msgr2=0x7f8ee40ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8ee95ed640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ee4144780 0x7f8ee40ae3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8ee95ed640 1 -- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ee4072680 msgr2=0x7f8ee40c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8ee95ed640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ee4072680 0x7f8ee40c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8ee95ed640 1 -- 172.21.15.98:0/3267937189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee415a020 con 0x7f8ee40ceb00 2024-09-10T21:00:17.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8eea5ef640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ee4144780 0x7f8ee40ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:00:17.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8ee95ed640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee40ceb00 0x7f8ee4159ae0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000d980 tx=0x7f8ee000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:17.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8ed67fc640 1 -- 172.21.15.98:0/3267937189 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0017070 con 0x7f8ee40ceb00 2024-09-10T21:00:17.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8ed67fc640 1 -- 172.21.15.98:0/3267937189 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ee0009d20 con 0x7f8ee40ceb00 2024-09-10T21:00:17.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8eec052640 1 -- 172.21.15.98:0/3267937189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee40ad930 con 0x7f8ee40ceb00 2024-09-10T21:00:17.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8ed67fc640 1 -- 172.21.15.98:0/3267937189 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00106c0 con 0x7f8ee40ceb00 2024-09-10T21:00:17.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.276+0000 7f8eec052640 1 -- 172.21.15.98:0/3267937189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee40adbd0 con 0x7f8ee40ceb00 2024-09-10T21:00:17.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.277+0000 7f8eec052640 1 -- 172.21.15.98:0/3267937189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee40cf420 con 0x7f8ee40ceb00 2024-09-10T21:00:17.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.278+0000 7f8ed67fc640 1 -- 172.21.15.98:0/3267937189 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8ee0010860 con 0x7f8ee40ceb00 2024-09-10T21:00:17.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.278+0000 7f8ed67fc640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8ec8077fa0 0x7f8ec807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:17.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.278+0000 7f8ed67fc640 1 -- 172.21.15.98:0/3267937189 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f8ee009a230 con 0x7f8ee40ceb00 2024-09-10T21:00:17.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.279+0000 7f8ee9dee640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8ec8077fa0 0x7f8ec807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:17.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.279+0000 7f8ee9dee640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8ec8077fa0 0x7f8ec807a460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8034000 tx=0x7f8ed8004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:17.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.282+0000 7f8ed67fc640 1 -- 172.21.15.98:0/3267937189 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8ee0062590 con 0x7f8ee40ceb00 2024-09-10T21:00:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.492+0000 7f8eec052640 1 -- 172.21.15.98:0/3267937189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ee40b4d40 con 0x7f8ee40ceb00 2024-09-10T21:00:17.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.493+0000 7f8ed67fc640 1 -- 172.21.15.98:0/3267937189 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5231 (secure 0 0 0) 0x7f8ee0066fc0 con 0x7f8ee40ceb00 2024-09-10T21:00:17.497 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:17.497 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":23,"btime":"2024-09-10T21:00:12:974795+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/1700293059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":1700293059},{"type":"v1","addr":"172.21.15.98:6834","nonce":1700293059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4048376378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4048376378},{"type":"v1","addr":"172.21.15.190:6835","nonce":4048376378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/2245019178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2245019178},{"type":"v1","addr":"172.21.15.98:6835","nonce":2245019178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T20:59:46.344123+0000","modified":"2024-09-10T21:00:12.974787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4834},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4834":{"gid":4834,"name":"b","rank":0,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.190:6837/3087107175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3087107175},{"type":"v1","addr":"172.21.15.190:6837","nonce":3087107175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4834,"qdb_cluster":[4834]},"id":2}]} 2024-09-10T21:00:17.497 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 23 2024-09-10T21:00:17.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.495+0000 7f8eec052640 1 -- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8ec8077fa0 msgr2=0x7f8ec807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.495+0000 7f8eec052640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8ec8077fa0 0x7f8ec807a460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8034000 tx=0x7f8ed8004730 comp rx=0 tx=0).stop 2024-09-10T21:00:17.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.495+0000 7f8eec052640 1 -- 172.21.15.98:0/3267937189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee40ceb00 msgr2=0x7f8ee4159ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.495+0000 7f8eec052640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee40ceb00 0x7f8ee4159ae0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000d980 tx=0x7f8ee000de40 comp rx=0 tx=0).stop 2024-09-10T21:00:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.495+0000 7f8eec052640 1 -- 172.21.15.98:0/3267937189 shutdown_connections 2024-09-10T21:00:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.495+0000 7f8eec052640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8ec8077fa0 0x7f8ec807a460 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.495+0000 7f8eec052640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8ee4144780 0x7f8ee40ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.495+0000 7f8eec052640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee40ceb00 0x7f8ee4159ae0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.495+0000 7f8eec052640 1 --2- 172.21.15.98:0/3267937189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ee4072680 0x7f8ee40c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.495+0000 7f8eec052640 1 -- 172.21.15.98:0/3267937189 >> 172.21.15.98:0/3267937189 conn(0x7f8ee40b1760 msgr2=0x7f8ee415b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.495+0000 7f8eec052640 1 -- 172.21.15.98:0/3267937189 shutdown_connections 2024-09-10T21:00:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.495+0000 7f8eec052640 1 -- 172.21.15.98:0/3267937189 wait complete. 2024-09-10T21:00:17.510 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:00:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.606+0000 7f96775e2640 1 -- 172.21.15.98:0/2954285361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9670073780 msgr2=0x7f9670073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.606+0000 7f96775e2640 1 --2- 172.21.15.98:0/2954285361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9670073780 0x7f9670073b60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9660005fd0 tx=0x7f966002aa00 comp rx=0 tx=0).stop 2024-09-10T21:00:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.606+0000 7f96775e2640 1 -- 172.21.15.98:0/2954285361 shutdown_connections 2024-09-10T21:00:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.606+0000 7f96775e2640 1 --2- 172.21.15.98:0/2954285361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f967006f140 0x7f967006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.606+0000 7f96775e2640 1 --2- 172.21.15.98:0/2954285361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9670073780 0x7f9670073b60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.606+0000 7f96775e2640 1 -- 172.21.15.98:0/2954285361 >> 172.21.15.98:0/2954285361 conn(0x7f96700b1760 msgr2=0x7f96700b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.606+0000 7f96775e2640 1 -- 172.21.15.98:0/2954285361 shutdown_connections 2024-09-10T21:00:17.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.606+0000 7f96775e2640 1 -- 172.21.15.98:0/2954285361 wait complete. 2024-09-10T21:00:17.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.607+0000 7f96775e2640 1 Processor -- start 2024-09-10T21:00:17.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.607+0000 7f96775e2640 1 -- start start 2024-09-10T21:00:17.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.608+0000 7f96775e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f967006f140 0x7f967013d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:17.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.608+0000 7f96775e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9670073780 0x7f967015aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:17.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.608+0000 7f96775e2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f967015c470 0x7f967015b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:17.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.608+0000 7f96775e2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9670142260 con 0x7f967015c470 2024-09-10T21:00:17.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.608+0000 7f96775e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f96701420e0 con 0x7f967006f140 2024-09-10T21:00:17.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.608+0000 7f96775e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f967006fb10 con 0x7f9670073780 2024-09-10T21:00:17.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.608+0000 7f9675b7f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f967015c470 0x7f967015b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:17.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.608+0000 7f9675b7f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f967015c470 0x7f967015b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41366/0 (socket says 172.21.15.98:41366) 2024-09-10T21:00:17.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.608+0000 7f9675b7f640 1 -- 172.21.15.98:0/3694520603 learned_addr learned my addr 172.21.15.98:0/3694520603 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:17.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f9674b7d640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9670073780 0x7f967015aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:17.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f967537e640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f967006f140 0x7f967013d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:17.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f9675b7f640 1 -- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9670073780 msgr2=0x7f967015aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f9675b7f640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9670073780 0x7f967015aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f9675b7f640 1 -- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f967006f140 msgr2=0x7f967013d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f9675b7f640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f967006f140 0x7f967013d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f9675b7f640 1 -- 172.21.15.98:0/3694520603 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9670155dc0 con 0x7f967015c470 2024-09-10T21:00:17.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f9674b7d640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9670073780 0x7f967015aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:00:17.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f967537e640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f967006f140 0x7f967013d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:00:17.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f9675b7f640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f967015c470 0x7f967015b1e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9668009fd0 tx=0x7f966800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:17.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f966dffb640 1 -- 172.21.15.98:0/3694520603 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668011830 con 0x7f967015c470 2024-09-10T21:00:17.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f966dffb640 1 -- 172.21.15.98:0/3694520603 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9668016070 con 0x7f967015c470 2024-09-10T21:00:17.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f96775e2640 1 -- 172.21.15.98:0/3694520603 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96701560a0 con 0x7f967015c470 2024-09-10T21:00:17.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f966dffb640 1 -- 172.21.15.98:0/3694520603 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668011d40 con 0x7f967015c470 2024-09-10T21:00:17.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.609+0000 7f96775e2640 1 -- 172.21.15.98:0/3694520603 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96700b0940 con 0x7f967015c470 2024-09-10T21:00:17.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.610+0000 7f96775e2640 1 -- 172.21.15.98:0/3694520603 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96700700c0 con 0x7f967015c470 2024-09-10T21:00:17.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.611+0000 7f966dffb640 1 -- 172.21.15.98:0/3694520603 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f966801b460 con 0x7f967015c470 2024-09-10T21:00:17.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.611+0000 7f966dffb640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f965c077f70 0x7f965c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:17.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.611+0000 7f966dffb640 1 -- 172.21.15.98:0/3694520603 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f9668099bd0 con 0x7f967015c470 2024-09-10T21:00:17.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.611+0000 7f967537e640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f965c077f70 0x7f965c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:17.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.612+0000 7f967537e640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f965c077f70 0x7f965c07a430 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9660035000 tx=0x7f966003a040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:17.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.615+0000 7f966dffb640 1 -- 172.21.15.98:0/3694520603 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9668062580 con 0x7f967015c470 2024-09-10T21:00:17.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.789+0000 7f96775e2640 1 -- 172.21.15.98:0/3694520603 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96700ce540 con 0x7f967015c470 2024-09-10T21:00:17.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.790+0000 7f966dffb640 1 -- 172.21.15.98:0/3694520603 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17143 (secure 0 0 0) 0x7f9668061e80 con 0x7f967015c470 2024-09-10T21:00:17.794 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:17.794 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":41,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T21:00:08.963642+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T20:59:44.162419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T20:59:45.314201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6835/4202850803":"2024-09-11T21:00:08.963614+0000","172.21.15.98:6832/4202850803":"2024-09-11T21:00:08.963614+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:00:17.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.792+0000 7f96775e2640 1 -- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f965c077f70 msgr2=0x7f965c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.792+0000 7f96775e2640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f965c077f70 0x7f965c07a430 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9660035000 tx=0x7f966003a040 comp rx=0 tx=0).stop 2024-09-10T21:00:17.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.792+0000 7f96775e2640 1 -- 172.21.15.98:0/3694520603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f967015c470 msgr2=0x7f967015b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.792+0000 7f96775e2640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f967015c470 0x7f967015b1e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9668009fd0 tx=0x7f966800b8e0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.792+0000 7f96775e2640 1 -- 172.21.15.98:0/3694520603 shutdown_connections 2024-09-10T21:00:17.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.792+0000 7f96775e2640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f965c077f70 0x7f965c07a430 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.792+0000 7f96775e2640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f967015c470 0x7f967015b1e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.792+0000 7f96775e2640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9670073780 0x7f967015aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.792+0000 7f96775e2640 1 --2- 172.21.15.98:0/3694520603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f967006f140 0x7f967013d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.792+0000 7f96775e2640 1 -- 172.21.15.98:0/3694520603 >> 172.21.15.98:0/3694520603 conn(0x7f96700b1760 msgr2=0x7f96700af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:17.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.792+0000 7f96775e2640 1 -- 172.21.15.98:0/3694520603 shutdown_connections 2024-09-10T21:00:17.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.792+0000 7f96775e2640 1 -- 172.21.15.98:0/3694520603 wait complete. 2024-09-10T21:00:17.807 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:00:17.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.904+0000 7f316baa9640 1 -- 172.21.15.98:0/1865057604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31640ccc10 msgr2=0x7f3164141650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.904+0000 7f316baa9640 1 --2- 172.21.15.98:0/1865057604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31640ccc10 0x7f3164141650 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f315c009ca0 tx=0x7f315c02aac0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.904+0000 7f316baa9640 1 -- 172.21.15.98:0/1865057604 shutdown_connections 2024-09-10T21:00:17.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.904+0000 7f316baa9640 1 --2- 172.21.15.98:0/1865057604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31640ccc10 0x7f3164141650 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.904+0000 7f316baa9640 1 --2- 172.21.15.98:0/1865057604 >> v2:172.21.15.190:3300/0 conn(0x7f316413ed50 0x7f316406f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.904+0000 7f316baa9640 1 -- 172.21.15.98:0/1865057604 >> 172.21.15.98:0/1865057604 conn(0x7f31640b1760 msgr2=0x7f31640b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:17.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.904+0000 7f316baa9640 1 -- 172.21.15.98:0/1865057604 shutdown_connections 2024-09-10T21:00:17.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.904+0000 7f316baa9640 1 -- 172.21.15.98:0/1865057604 wait complete. 2024-09-10T21:00:17.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.905+0000 7f316baa9640 1 Processor -- start 2024-09-10T21:00:17.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.905+0000 7f316baa9640 1 -- start start 2024-09-10T21:00:17.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f316baa9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3164073db0 0x7f316413a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:17.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f316baa9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31640ccc10 0x7f31641398d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:17.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f316baa9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f316413ed50 0x7f31641e8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:17.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f316baa9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3164141d10 con 0x7f31640ccc10 2024-09-10T21:00:17.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f316baa9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3164141b90 con 0x7f3164073db0 2024-09-10T21:00:17.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f316baa9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3164141e90 con 0x7f316413ed50 2024-09-10T21:00:17.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f3169044640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31640ccc10 0x7f31641398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:17.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f3169044640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31640ccc10 0x7f31641398d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41372/0 (socket says 172.21.15.98:41372) 2024-09-10T21:00:17.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f3169044640 1 -- 172.21.15.98:0/1220348010 learned_addr learned my addr 172.21.15.98:0/1220348010 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:17.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f316a046640 1 --2- 172.21.15.98:0/1220348010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f316413ed50 0x7f31641e8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:17.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f3169845640 1 --2- 172.21.15.98:0/1220348010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3164073db0 0x7f316413a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:17.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f3169044640 1 -- 172.21.15.98:0/1220348010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f316413ed50 msgr2=0x7f31641e8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f3169044640 1 --2- 172.21.15.98:0/1220348010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f316413ed50 0x7f31641e8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f3169044640 1 -- 172.21.15.98:0/1220348010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3164073db0 msgr2=0x7f316413a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:17.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.906+0000 7f3169044640 1 --2- 172.21.15.98:0/1220348010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3164073db0 0x7f316413a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:17.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.907+0000 7f3169044640 1 -- 172.21.15.98:0/1220348010 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31641f5810 con 0x7f31640ccc10 2024-09-10T21:00:17.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.907+0000 7f3169044640 1 --2- 172.21.15.98:0/1220348010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31640ccc10 0x7f31641398d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f315400c450 tx=0x7f315400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:17.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.907+0000 7f315a7fc640 1 -- 172.21.15.98:0/1220348010 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31540052d0 con 0x7f31640ccc10 2024-09-10T21:00:17.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.907+0000 7f315a7fc640 1 -- 172.21.15.98:0/1220348010 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3154005470 con 0x7f31640ccc10 2024-09-10T21:00:17.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.907+0000 7f316baa9640 1 -- 172.21.15.98:0/1220348010 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31640b2d70 con 0x7f31640ccc10 2024-09-10T21:00:17.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.907+0000 7f315a7fc640 1 -- 172.21.15.98:0/1220348010 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154012870 con 0x7f31640ccc10 2024-09-10T21:00:17.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.907+0000 7f316baa9640 1 -- 172.21.15.98:0/1220348010 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31640b3330 con 0x7f31640ccc10 2024-09-10T21:00:17.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.908+0000 7f316baa9640 1 -- 172.21.15.98:0/1220348010 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31640b2b60 con 0x7f31640ccc10 2024-09-10T21:00:17.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.908+0000 7f315a7fc640 1 -- 172.21.15.98:0/1220348010 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f31540035f0 con 0x7f31640ccc10 2024-09-10T21:00:17.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.909+0000 7f315a7fc640 1 --2- 172.21.15.98:0/1220348010 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3148077f70 0x7f314807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:17.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.909+0000 7f315a7fc640 1 -- 172.21.15.98:0/1220348010 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6187+0+0 (secure 0 0 0) 0x7f3154099910 con 0x7f31640ccc10 2024-09-10T21:00:17.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.909+0000 7f3169845640 1 --2- 172.21.15.98:0/1220348010 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3148077f70 0x7f314807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:17.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.910+0000 7f3169845640 1 --2- 172.21.15.98:0/1220348010 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3148077f70 0x7f314807a430 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3164141340 tx=0x7f3160005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:17.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:17.913+0000 7f315a7fc640 1 -- 172.21.15.98:0/1220348010 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f316406f660 con 0x7f31640ccc10 2024-09-10T21:00:18.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.121+0000 7f316baa9640 1 -- 172.21.15.98:0/1220348010 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3164141650 con 0x7f31640ccc10 2024-09-10T21:00:18.126 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-10T21:00:18.122+0000 7fb4184a9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:00:18.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.124+0000 7f315a7fc640 1 -- 172.21.15.98:0/1220348010 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) ==== 161+0+0 (secure 0 0 0) 0x7f3154061be0 con 0x7f31640ccc10 2024-09-10T21:00:18.128 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:00:18.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.126+0000 7f316baa9640 1 -- 172.21.15.98:0/1220348010 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3148077f70 msgr2=0x7f314807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:18.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.126+0000 7f316baa9640 1 --2- 172.21.15.98:0/1220348010 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3148077f70 0x7f314807a430 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3164141340 tx=0x7f3160005f90 comp rx=0 tx=0).stop 2024-09-10T21:00:18.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.126+0000 7f316baa9640 1 -- 172.21.15.98:0/1220348010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31640ccc10 msgr2=0x7f31641398d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:18.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.126+0000 7f316baa9640 1 --2- 172.21.15.98:0/1220348010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31640ccc10 0x7f31641398d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f315400c450 tx=0x7f315400c910 comp rx=0 tx=0).stop 2024-09-10T21:00:18.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.126+0000 7f316baa9640 1 -- 172.21.15.98:0/1220348010 shutdown_connections 2024-09-10T21:00:18.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.126+0000 7f316baa9640 1 --2- 172.21.15.98:0/1220348010 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f3148077f70 0x7f314807a430 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:18.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.126+0000 7f316baa9640 1 --2- 172.21.15.98:0/1220348010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f316413ed50 0x7f31641e8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:18.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.126+0000 7f316baa9640 1 --2- 172.21.15.98:0/1220348010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31640ccc10 0x7f31641398d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:18.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.126+0000 7f316baa9640 1 --2- 172.21.15.98:0/1220348010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3164073db0 0x7f316413a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:18.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.126+0000 7f316baa9640 1 -- 172.21.15.98:0/1220348010 >> 172.21.15.98:0/1220348010 conn(0x7f31640b1760 msgr2=0x7f316413b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:18.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.126+0000 7f316baa9640 1 -- 172.21.15.98:0/1220348010 shutdown_connections 2024-09-10T21:00:18.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.127+0000 7f316baa9640 1 -- 172.21.15.98:0/1220348010 wait complete. 2024-09-10T21:00:18.137 INFO:tasks.ceph.mds.b.smithi190.stderr: -384> 2024-09-10T21:00:14.987+0000 7f78b7a08640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:00:18.141 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:00:18.165 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2024-09-10T21:00:18.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.240+0000 7ff02650e640 1 -- 172.21.15.98:0/3896180179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff008003a80 msgr2=0x7ff008003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:18.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.240+0000 7ff02650e640 1 --2- 172.21.15.98:0/3896180179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff008003a80 0x7ff008003ea0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff010006fd0 tx=0x7ff008005830 comp rx=0 tx=0).stop 2024-09-10T21:00:18.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.240+0000 7ff024aab640 1 -- 172.21.15.98:0/3896180179 reap_dead start 2024-09-10T21:00:18.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.240+0000 7ff02650e640 1 -- 172.21.15.98:0/3896180179 shutdown_connections 2024-09-10T21:00:18.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.240+0000 7ff02650e640 1 -- 172.21.15.98:0/3896180179 >> 172.21.15.98:0/3896180179 conn(0x7ff0200d3f70 msgr2=0x7ff0200d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:18.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.240+0000 7ff02650e640 1 -- 172.21.15.98:0/3896180179 shutdown_connections 2024-09-10T21:00:18.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.240+0000 7ff02650e640 1 -- 172.21.15.98:0/3896180179 wait complete. 2024-09-10T21:00:18.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.241+0000 7ff02650e640 1 Processor -- start 2024-09-10T21:00:18.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.241+0000 7ff02650e640 1 -- start start 2024-09-10T21:00:18.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff02650e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff008003a80 0x7ff0200cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:18.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff02650e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0201f83c0 0x7ff02015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:18.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff02650e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0201f8840 0x7ff02015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:18.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff02650e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff020166f70 con 0x7ff008003a80 2024-09-10T21:00:18.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff02650e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff020166df0 con 0x7ff0201f8840 2024-09-10T21:00:18.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff02650e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff0201670f0 con 0x7ff0201f83c0 2024-09-10T21:00:18.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff01ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff008003a80 0x7ff0200cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:18.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff01ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff008003a80 0x7ff0200cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41388/0 (socket says 172.21.15.98:41388) 2024-09-10T21:00:18.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff01ffff640 1 -- 172.21.15.98:0/2515860177 learned_addr learned my addr 172.21.15.98:0/2515860177 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:18.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff024aab640 1 --2- 172.21.15.98:0/2515860177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0201f8840 0x7ff02015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:18.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff01f7fe640 1 --2- 172.21.15.98:0/2515860177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0201f83c0 0x7ff02015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:18.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff01ffff640 1 -- 172.21.15.98:0/2515860177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0201f83c0 msgr2=0x7ff02015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:18.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff01ffff640 1 --2- 172.21.15.98:0/2515860177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0201f83c0 0x7ff02015aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:18.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff01ffff640 1 -- 172.21.15.98:0/2515860177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0201f8840 msgr2=0x7ff02015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:18.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff01ffff640 1 --2- 172.21.15.98:0/2515860177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0201f8840 0x7ff02015b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:18.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.242+0000 7ff01ffff640 1 -- 172.21.15.98:0/2515860177 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0200cd1c0 con 0x7ff008003a80 2024-09-10T21:00:18.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.243+0000 7ff01ffff640 1 --2- 172.21.15.98:0/2515860177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff008003a80 0x7ff0200cdde0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff01000c350 tx=0x7ff010006c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:18.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.243+0000 7ff01cff9640 1 -- 172.21.15.98:0/2515860177 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010031040 con 0x7ff008003a80 2024-09-10T21:00:18.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.243+0000 7ff01cff9640 1 -- 172.21.15.98:0/2515860177 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0100035f0 con 0x7ff008003a80 2024-09-10T21:00:18.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.243+0000 7ff01cff9640 1 -- 172.21.15.98:0/2515860177 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010003a00 con 0x7ff008003a80 2024-09-10T21:00:18.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.243+0000 7ff02650e640 1 -- 172.21.15.98:0/2515860177 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0200cd4a0 con 0x7ff008003a80 2024-09-10T21:00:18.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.243+0000 7ff02650e640 1 -- 172.21.15.98:0/2515860177 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0200cb450 con 0x7ff008003a80 2024-09-10T21:00:18.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.244+0000 7ff02650e640 1 -- 172.21.15.98:0/2515860177 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0200d24e0 con 0x7ff008003a80 2024-09-10T21:00:18.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.244+0000 7ff01cff9640 1 -- 172.21.15.98:0/2515860177 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff010048030 con 0x7ff008003a80 2024-09-10T21:00:18.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.245+0000 7ff01cff9640 1 --2- 172.21.15.98:0/2515860177 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff00c077fa0 0x7ff00c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:18.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.245+0000 7ff01cff9640 1 -- 172.21.15.98:0/2515860177 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6273+0+0 (secure 0 0 0) 0x7ff0100b9b80 con 0x7ff008003a80 2024-09-10T21:00:18.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.245+0000 7ff01f7fe640 1 --2- 172.21.15.98:0/2515860177 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff00c077fa0 0x7ff00c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:18.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.245+0000 7ff01f7fe640 1 --2- 172.21.15.98:0/2515860177 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff00c077fa0 0x7ff00c07a460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff0200a1ce0 tx=0x7ff014006d90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:18.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.249+0000 7ff01cff9640 1 -- 172.21.15.98:0/2515860177 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff0100824d0 con 0x7ff008003a80 2024-09-10T21:00:18.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:18.459+0000 7ff02650e640 1 -- 172.21.15.98:0/2515860177 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff02008dfa0 con 0x7ff008003a80 2024-09-10T21:00:19.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.125+0000 7ff01cff9640 1 -- 172.21.15.98:0/2515860177 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7ff010081dd0 con 0x7ff008003a80 2024-09-10T21:00:19.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.128+0000 7ff02650e640 1 -- 172.21.15.98:0/2515860177 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff00c077fa0 msgr2=0x7ff00c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:19.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.128+0000 7ff02650e640 1 --2- 172.21.15.98:0/2515860177 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff00c077fa0 0x7ff00c07a460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff0200a1ce0 tx=0x7ff014006d90 comp rx=0 tx=0).stop 2024-09-10T21:00:19.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.128+0000 7ff02650e640 1 -- 172.21.15.98:0/2515860177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff008003a80 msgr2=0x7ff0200cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:19.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.128+0000 7ff02650e640 1 --2- 172.21.15.98:0/2515860177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff008003a80 0x7ff0200cdde0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff01000c350 tx=0x7ff010006c80 comp rx=0 tx=0).stop 2024-09-10T21:00:19.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.128+0000 7ff02650e640 1 -- 172.21.15.98:0/2515860177 shutdown_connections 2024-09-10T21:00:19.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.128+0000 7ff02650e640 1 --2- 172.21.15.98:0/2515860177 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7ff00c077fa0 0x7ff00c07a460 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:19.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.128+0000 7ff02650e640 1 --2- 172.21.15.98:0/2515860177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0201f8840 0x7ff02015b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:19.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.128+0000 7ff02650e640 1 --2- 172.21.15.98:0/2515860177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0201f83c0 0x7ff02015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:19.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.128+0000 7ff02650e640 1 --2- 172.21.15.98:0/2515860177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff008003a80 0x7ff0200cdde0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:19.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.128+0000 7ff02650e640 1 -- 172.21.15.98:0/2515860177 >> 172.21.15.98:0/2515860177 conn(0x7ff0200d3f70 msgr2=0x7ff02006eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:19.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.128+0000 7ff02650e640 1 -- 172.21.15.98:0/2515860177 shutdown_connections 2024-09-10T21:00:19.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.128+0000 7ff02650e640 1 -- 172.21.15.98:0/2515860177 wait complete. 2024-09-10T21:00:19.145 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:00:19.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.250+0000 7f36f14ef640 1 -- 172.21.15.98:0/947350645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36ec0d2a20 msgr2=0x7f36ec0cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:19.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.250+0000 7f36f14ef640 1 --2- 172.21.15.98:0/947350645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36ec0d2a20 0x7f36ec0cea40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009830 tx=0x7f36e402a4d0 comp rx=0 tx=0).stop 2024-09-10T21:00:19.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.250+0000 7f36f14ef640 1 -- 172.21.15.98:0/947350645 shutdown_connections 2024-09-10T21:00:19.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.250+0000 7f36f14ef640 1 --2- 172.21.15.98:0/947350645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36ec0d2a20 0x7f36ec0cea40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:19.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.250+0000 7f36f14ef640 1 --2- 172.21.15.98:0/947350645 >> v2:172.21.15.190:3300/0 conn(0x7f36ec0d4af0 0x7f36ec0d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:19.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.250+0000 7f36f14ef640 1 -- 172.21.15.98:0/947350645 >> 172.21.15.98:0/947350645 conn(0x7f36ec0d3f70 msgr2=0x7f36ec0d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:19.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.250+0000 7f36f14ef640 1 -- 172.21.15.98:0/947350645 shutdown_connections 2024-09-10T21:00:19.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.250+0000 7f36f14ef640 1 -- 172.21.15.98:0/947350645 wait complete. 2024-09-10T21:00:19.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.251+0000 7f36f14ef640 1 Processor -- start 2024-09-10T21:00:19.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.251+0000 7f36f14ef640 1 -- start start 2024-09-10T21:00:19.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36f14ef640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36ec0d2a20 0x7f36ec1583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:19.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36f14ef640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36ec0d4af0 0x7f36ec0c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:19.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36f14ef640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36ec15e060 0x7f36ec15f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:19.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36f14ef640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec0cf100 con 0x7f36ec0d4af0 2024-09-10T21:00:19.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36f14ef640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec0cef80 con 0x7f36ec15e060 2024-09-10T21:00:19.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36f14ef640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f36ec0cf280 con 0x7f36ec0d2a20 2024-09-10T21:00:19.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36ea7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36ec0d4af0 0x7f36ec0c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:19.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36ea7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36ec0d4af0 0x7f36ec0c87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41416/0 (socket says 172.21.15.98:41416) 2024-09-10T21:00:19.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36ea7fc640 1 -- 172.21.15.98:0/293876770 learned_addr learned my addr 172.21.15.98:0/293876770 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:19.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36eb7fe640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36ec15e060 0x7f36ec15f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:19.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36eaffd640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36ec0d2a20 0x7f36ec1583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:19.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36ea7fc640 1 -- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36ec0d2a20 msgr2=0x7f36ec1583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:19.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36ea7fc640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36ec0d2a20 0x7f36ec1583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:19.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36ea7fc640 1 -- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36ec15e060 msgr2=0x7f36ec15f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:19.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36ea7fc640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36ec15e060 0x7f36ec15f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:19.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36ea7fc640 1 -- 172.21.15.98:0/293876770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec15e7e0 con 0x7f36ec0d4af0 2024-09-10T21:00:19.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36eb7fe640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36ec15e060 0x7f36ec15f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:19.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36eaffd640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36ec0d2a20 0x7f36ec1583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:19.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.252+0000 7f36ea7fc640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36ec0d4af0 0x7f36ec0c87a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f36e000c9a0 tx=0x7f36e000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:19.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.253+0000 7f36d37fe640 1 -- 172.21.15.98:0/293876770 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0007bb0 con 0x7f36ec0d4af0 2024-09-10T21:00:19.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.253+0000 7f36d37fe640 1 -- 172.21.15.98:0/293876770 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36e0005020 con 0x7f36ec0d4af0 2024-09-10T21:00:19.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.253+0000 7f36d37fe640 1 -- 172.21.15.98:0/293876770 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0007e60 con 0x7f36ec0d4af0 2024-09-10T21:00:19.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.253+0000 7f36f14ef640 1 -- 172.21.15.98:0/293876770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ec15eac0 con 0x7f36ec0d4af0 2024-09-10T21:00:19.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.253+0000 7f36f14ef640 1 -- 172.21.15.98:0/293876770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ec15f080 con 0x7f36ec0d4af0 2024-09-10T21:00:19.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.254+0000 7f36f14ef640 1 -- 172.21.15.98:0/293876770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36ec0cf730 con 0x7f36ec0d4af0 2024-09-10T21:00:19.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.254+0000 7f36d37fe640 1 -- 172.21.15.98:0/293876770 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f36e0005470 con 0x7f36ec0d4af0 2024-09-10T21:00:19.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.255+0000 7f36d37fe640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f36d8077f70 0x7f36d807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:19.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.255+0000 7f36d37fe640 1 -- 172.21.15.98:0/293876770 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6273+0+0 (secure 0 0 0) 0x7f36e009a440 con 0x7f36ec0d4af0 2024-09-10T21:00:19.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.255+0000 7f36eaffd640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f36d8077f70 0x7f36d807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:19.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.256+0000 7f36eaffd640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f36d8077f70 0x7f36d807a430 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0012e0 tx=0x7f36dc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:19.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.259+0000 7f36d37fe640 1 -- 172.21.15.98:0/293876770 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f36e0014030 con 0x7f36ec0d4af0 2024-09-10T21:00:19.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:19.434+0000 7f36f14ef640 1 -- 172.21.15.98:0/293876770 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f36ec08dfa0 con 0x7f36ec0d4af0 2024-09-10T21:00:20.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.435+0000 7f36d37fe640 1 -- 172.21.15.98:0/293876770 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) ==== 181+0+0 (secure 0 0 0) 0x7f36e0062d90 con 0x7f36ec0d4af0 2024-09-10T21:00:20.439 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:00:20.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.439+0000 7f36f14ef640 1 -- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f36d8077f70 msgr2=0x7f36d807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:20.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.439+0000 7f36f14ef640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f36d8077f70 0x7f36d807a430 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0012e0 tx=0x7f36dc009040 comp rx=0 tx=0).stop 2024-09-10T21:00:20.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.439+0000 7f36f14ef640 1 -- 172.21.15.98:0/293876770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36ec0d4af0 msgr2=0x7f36ec0c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:20.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.439+0000 7f36f14ef640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36ec0d4af0 0x7f36ec0c87a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f36e000c9a0 tx=0x7f36e000ce60 comp rx=0 tx=0).stop 2024-09-10T21:00:20.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.439+0000 7f36f14ef640 1 -- 172.21.15.98:0/293876770 shutdown_connections 2024-09-10T21:00:20.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.439+0000 7f36f14ef640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f36d8077f70 0x7f36d807a430 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:20.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.439+0000 7f36f14ef640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36ec15e060 0x7f36ec15f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:20.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.439+0000 7f36f14ef640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36ec0d4af0 0x7f36ec0c87a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:20.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.439+0000 7f36f14ef640 1 --2- 172.21.15.98:0/293876770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36ec0d2a20 0x7f36ec1583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:20.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.439+0000 7f36f14ef640 1 -- 172.21.15.98:0/293876770 >> 172.21.15.98:0/293876770 conn(0x7f36ec0d3f70 msgr2=0x7f36ec162140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:20.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.439+0000 7f36f14ef640 1 -- 172.21.15.98:0/293876770 shutdown_connections 2024-09-10T21:00:20.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.439+0000 7f36f14ef640 1 -- 172.21.15.98:0/293876770 wait complete. 2024-09-10T21:00:20.457 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:00:20.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.565+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/368017532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1d0004040 msgr2=0x7fd1d0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:20.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.565+0000 7fd1ee9a3640 1 --2- 172.21.15.98:0/368017532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1d0004040 0x7fd1d0004800 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c710 tx=0x7fd1d0005830 comp rx=0 tx=0).stop 2024-09-10T21:00:20.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.565+0000 7fd1ecf40640 1 -- 172.21.15.98:0/368017532 reap_dead start 2024-09-10T21:00:20.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.566+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/368017532 shutdown_connections 2024-09-10T21:00:20.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.566+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/368017532 >> 172.21.15.98:0/368017532 conn(0x7fd1e80b1760 msgr2=0x7fd1e80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:20.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.566+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/368017532 shutdown_connections 2024-09-10T21:00:20.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.566+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/368017532 wait complete. 2024-09-10T21:00:20.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.567+0000 7fd1ee9a3640 1 Processor -- start 2024-09-10T21:00:20.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.567+0000 7fd1ee9a3640 1 -- start start 2024-09-10T21:00:20.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1ee9a3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1d0004040 0x7fd1e81eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:20.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1ee9a3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1e81f5d90 0x7fd1e813cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:20.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1ee9a3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1e813d6b0 0x7fd1e813db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:20.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1ee9a3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e815c6d0 con 0x7fd1d0004040 2024-09-10T21:00:20.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1ee9a3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e815c550 con 0x7fd1e813d6b0 2024-09-10T21:00:20.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1ee9a3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e815c850 con 0x7fd1e81f5d90 2024-09-10T21:00:20.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1e7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1d0004040 0x7fd1e81eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:20.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1e7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1d0004040 0x7fd1e81eeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41858/0 (socket says 172.21.15.98:41858) 2024-09-10T21:00:20.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1e7fff640 1 -- 172.21.15.98:0/1969050311 learned_addr learned my addr 172.21.15.98:0/1969050311 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:20.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1e77fe640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1e81f5d90 0x7fd1e813cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:20.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1ecf40640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1e813d6b0 0x7fd1e813db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:20.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1e7fff640 1 -- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1e81f5d90 msgr2=0x7fd1e813cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:20.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1e7fff640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1e81f5d90 0x7fd1e813cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:20.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1e7fff640 1 -- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1e813d6b0 msgr2=0x7fd1e813db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:20.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1e7fff640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1e813d6b0 0x7fd1e813db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:20.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1e7fff640 1 -- 172.21.15.98:0/1969050311 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e80a9d70 con 0x7fd1d0004040 2024-09-10T21:00:20.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1e77fe640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1e81f5d90 0x7fd1e813cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:00:20.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1e7fff640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1d0004040 0x7fd1e81eeed0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8003850 tx=0x7fd1d8003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:20.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1e4ff9640 1 -- 172.21.15.98:0/1969050311 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8003e40 con 0x7fd1d0004040 2024-09-10T21:00:20.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1e4ff9640 1 -- 172.21.15.98:0/1969050311 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1d8018020 con 0x7fd1d0004040 2024-09-10T21:00:20.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1e4ff9640 1 -- 172.21.15.98:0/1969050311 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8011d40 con 0x7fd1d0004040 2024-09-10T21:00:20.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/1969050311 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e80a9fe0 con 0x7fd1d0004040 2024-09-10T21:00:20.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.568+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/1969050311 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e81f1c70 con 0x7fd1d0004040 2024-09-10T21:00:20.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.569+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/1969050311 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e8074360 con 0x7fd1d0004040 2024-09-10T21:00:20.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.570+0000 7fd1e4ff9640 1 -- 172.21.15.98:0/1969050311 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd1d8010040 con 0x7fd1d0004040 2024-09-10T21:00:20.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.571+0000 7fd1e4ff9640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd1d4077f70 0x7fd1d407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:20.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.571+0000 7fd1e4ff9640 1 -- 172.21.15.98:0/1969050311 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5835+0+0 (secure 0 0 0) 0x7fd1d809ce50 con 0x7fd1d0004040 2024-09-10T21:00:20.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.571+0000 7fd1e77fe640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd1d4077f70 0x7fd1d407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:20.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.572+0000 7fd1e77fe640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd1d4077f70 0x7fd1d407a430 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc032000 tx=0x7fd1dc003760 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:20.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.575+0000 7fd1e4ff9640 1 -- 172.21.15.98:0/1969050311 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd1e8074360 con 0x7fd1d0004040 2024-09-10T21:00:20.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:20.749+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/1969050311 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd1e8070e70 con 0x7fd1d0004040 2024-09-10T21:00:21.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.440+0000 7fd1e4ff9640 1 -- 172.21.15.98:0/1969050311 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) ==== 169+0+0 (secure 0 0 0) 0x7fd1e8070e70 con 0x7fd1d0004040 2024-09-10T21:00:21.444 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-09-10T21:00:21.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.443+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd1d4077f70 msgr2=0x7fd1d407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:21.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.443+0000 7fd1ee9a3640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd1d4077f70 0x7fd1d407a430 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc032000 tx=0x7fd1dc003760 comp rx=0 tx=0).stop 2024-09-10T21:00:21.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.443+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/1969050311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1d0004040 msgr2=0x7fd1e81eeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:21.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.443+0000 7fd1ee9a3640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1d0004040 0x7fd1e81eeed0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8003850 tx=0x7fd1d8003d10 comp rx=0 tx=0).stop 2024-09-10T21:00:21.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.443+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/1969050311 shutdown_connections 2024-09-10T21:00:21.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.443+0000 7fd1ee9a3640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fd1d4077f70 0x7fd1d407a430 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:21.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.443+0000 7fd1ee9a3640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1e813d6b0 0x7fd1e813db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:21.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.443+0000 7fd1ee9a3640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1e81f5d90 0x7fd1e813cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:21.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.443+0000 7fd1ee9a3640 1 --2- 172.21.15.98:0/1969050311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1d0004040 0x7fd1e81eeed0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:21.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.443+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/1969050311 >> 172.21.15.98:0/1969050311 conn(0x7fd1e80b1760 msgr2=0x7fd1e80afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:21.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.443+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/1969050311 shutdown_connections 2024-09-10T21:00:21.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.443+0000 7fd1ee9a3640 1 -- 172.21.15.98:0/1969050311 wait complete. 2024-09-10T21:00:21.460 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:00:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.564+0000 7f7933365640 1 -- 172.21.15.98:0/1549131634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7914004000 msgr2=0x7f7914004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.564+0000 7f7933365640 1 --2- 172.21.15.98:0/1549131634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7914004000 0x7f7914004800 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7920009b20 tx=0x7f7914005830 comp rx=0 tx=0).stop 2024-09-10T21:00:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.564+0000 7f7931902640 1 -- 172.21.15.98:0/1549131634 reap_dead start 2024-09-10T21:00:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.564+0000 7f7933365640 1 -- 172.21.15.98:0/1549131634 shutdown_connections 2024-09-10T21:00:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.564+0000 7f7933365640 1 -- 172.21.15.98:0/1549131634 >> 172.21.15.98:0/1549131634 conn(0x7f792c0b1760 msgr2=0x7f792c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.564+0000 7f7933365640 1 -- 172.21.15.98:0/1549131634 shutdown_connections 2024-09-10T21:00:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.564+0000 7f7933365640 1 -- 172.21.15.98:0/1549131634 wait complete. 2024-09-10T21:00:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.565+0000 7f7933365640 1 Processor -- start 2024-09-10T21:00:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.565+0000 7f7933365640 1 -- start start 2024-09-10T21:00:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7933365640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7914004000 0x7f792c073fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7933365640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f792c138f60 0x7f792c14ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7933365640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c14fd10 0x7f792c14f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7933365640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f792c0b4890 con 0x7f792c138f60 2024-09-10T21:00:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7933365640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f792c0b4710 con 0x7f7914004000 2024-09-10T21:00:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7933365640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f792c0b4a10 con 0x7f792c14fd10 2024-09-10T21:00:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7930900640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f792c138f60 0x7f792c14ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7930900640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f792c138f60 0x7f792c14ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41884/0 (socket says 172.21.15.98:41884) 2024-09-10T21:00:21.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7930900640 1 -- 172.21.15.98:0/2355807728 learned_addr learned my addr 172.21.15.98:0/2355807728 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:21.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7931101640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7914004000 0x7f792c073fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:21.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7930900640 1 -- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c14fd10 msgr2=0x7f792c14f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:21.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7931902640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c14fd10 0x7f792c14f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:21.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7930900640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c14fd10 0x7f792c14f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:21.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7930900640 1 -- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7914004000 msgr2=0x7f792c073fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:21.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7930900640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7914004000 0x7f792c073fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:21.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7930900640 1 -- 172.21.15.98:0/2355807728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c1e7f40 con 0x7f792c138f60 2024-09-10T21:00:21.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7931101640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7914004000 0x7f792c073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:21.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.566+0000 7f7931902640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c14fd10 0x7f792c14f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:21.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.567+0000 7f7930900640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f792c138f60 0x7f792c14ebc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7920009a70 tx=0x7f7920033650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:21.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.567+0000 7f7929ffb640 1 -- 172.21.15.98:0/2355807728 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920040020 con 0x7f792c138f60 2024-09-10T21:00:21.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.567+0000 7f7929ffb640 1 -- 172.21.15.98:0/2355807728 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7920005310 con 0x7f792c138f60 2024-09-10T21:00:21.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.567+0000 7f7933365640 1 -- 172.21.15.98:0/2355807728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792c1e81c0 con 0x7f792c138f60 2024-09-10T21:00:21.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.567+0000 7f7929ffb640 1 -- 172.21.15.98:0/2355807728 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920038040 con 0x7f792c138f60 2024-09-10T21:00:21.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.567+0000 7f7933365640 1 -- 172.21.15.98:0/2355807728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792c14f7e0 con 0x7f792c138f60 2024-09-10T21:00:21.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.568+0000 7f7933365640 1 -- 172.21.15.98:0/2355807728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792c0b3d10 con 0x7f792c138f60 2024-09-10T21:00:21.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.568+0000 7f7929ffb640 1 -- 172.21.15.98:0/2355807728 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f792003b070 con 0x7f792c138f60 2024-09-10T21:00:21.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.569+0000 7f7929ffb640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7918077f70 0x7f791807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:21.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.569+0000 7f7929ffb640 1 -- 172.21.15.98:0/2355807728 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f79200b9990 con 0x7f792c138f60 2024-09-10T21:00:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.569+0000 7f7931101640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7918077f70 0x7f791807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.570+0000 7f7931101640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7918077f70 0x7f791807a430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f791c0013d0 tx=0x7f791c006b20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.573+0000 7f7929ffb640 1 -- 172.21.15.98:0/2355807728 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f792c1e7f40 con 0x7f792c138f60 2024-09-10T21:00:21.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.747+0000 7f7933365640 1 -- 172.21.15.98:0/2355807728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f792c15d400 con 0x7f792c138f60 2024-09-10T21:00:21.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.750+0000 7f7929ffb640 1 -- 172.21.15.98:0/2355807728 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v29) ==== 71+0+0 (secure 0 0 0) 0x7f79200825b0 con 0x7f792c138f60 2024-09-10T21:00:21.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.752+0000 7f7933365640 1 -- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7918077f70 msgr2=0x7f791807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:21.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.752+0000 7f7933365640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7918077f70 0x7f791807a430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f791c0013d0 tx=0x7f791c006b20 comp rx=0 tx=0).stop 2024-09-10T21:00:21.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.753+0000 7f7933365640 1 -- 172.21.15.98:0/2355807728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f792c138f60 msgr2=0x7f792c14ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:21.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.753+0000 7f7933365640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f792c138f60 0x7f792c14ebc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7920009a70 tx=0x7f7920033650 comp rx=0 tx=0).stop 2024-09-10T21:00:21.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.753+0000 7f7933365640 1 -- 172.21.15.98:0/2355807728 shutdown_connections 2024-09-10T21:00:21.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.753+0000 7f7933365640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7918077f70 0x7f791807a430 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:21.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.753+0000 7f7933365640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f792c14fd10 0x7f792c14f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:21.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.753+0000 7f7933365640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f792c138f60 0x7f792c14ebc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:21.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.753+0000 7f7933365640 1 --2- 172.21.15.98:0/2355807728 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7914004000 0x7f792c073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:21.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.753+0000 7f7933365640 1 -- 172.21.15.98:0/2355807728 >> 172.21.15.98:0/2355807728 conn(0x7f792c0b1760 msgr2=0x7f792c0b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:21.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.753+0000 7f7933365640 1 -- 172.21.15.98:0/2355807728 shutdown_connections 2024-09-10T21:00:21.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.753+0000 7f7933365640 1 -- 172.21.15.98:0/2355807728 wait complete. 2024-09-10T21:00:21.768 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create' 2024-09-10T21:00:21.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.865+0000 7f19917ec640 1 -- 172.21.15.98:0/3988335558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c161560 msgr2=0x7f198c0d2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:21.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.865+0000 7f19917ec640 1 --2- 172.21.15.98:0/3988335558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c161560 0x7f198c0d2360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f197c005fd0 tx=0x7f197c02a540 comp rx=0 tx=0).stop 2024-09-10T21:00:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.865+0000 7f19917ec640 1 -- 172.21.15.98:0/3988335558 shutdown_connections 2024-09-10T21:00:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.865+0000 7f19917ec640 1 --2- 172.21.15.98:0/3988335558 >> v2:172.21.15.190:3300/0 conn(0x7f198c0b55a0 0x7f198c0d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.865+0000 7f19917ec640 1 --2- 172.21.15.98:0/3988335558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c161560 0x7f198c0d2360 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.865+0000 7f19917ec640 1 -- 172.21.15.98:0/3988335558 >> 172.21.15.98:0/3988335558 conn(0x7f198c09d170 msgr2=0x7f198c0d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.865+0000 7f19917ec640 1 -- 172.21.15.98:0/3988335558 shutdown_connections 2024-09-10T21:00:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.865+0000 7f19917ec640 1 -- 172.21.15.98:0/3988335558 wait complete. 2024-09-10T21:00:21.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.866+0000 7f19917ec640 1 Processor -- start 2024-09-10T21:00:21.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.866+0000 7f19917ec640 1 -- start start 2024-09-10T21:00:21.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f19917ec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c0b55a0 0x7f198c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:21.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f19917ec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f198c0d01b0 0x7f198c0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:21.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f19917ec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198c161560 0x7f198c1f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:21.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f19917ec640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f198c0d48d0 con 0x7f198c161560 2024-09-10T21:00:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f19917ec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f198c0b4c50 con 0x7f198c0d01b0 2024-09-10T21:00:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f19917ec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f198c0cf050 con 0x7f198c0b55a0 2024-09-10T21:00:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f198b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198c161560 0x7f198c1f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f198b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198c161560 0x7f198c1f79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41898/0 (socket says 172.21.15.98:41898) 2024-09-10T21:00:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f198b7fe640 1 -- 172.21.15.98:0/2223204730 learned_addr learned my addr 172.21.15.98:0/2223204730 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f198affd640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c0b55a0 0x7f198c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f1983fff640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f198c0d01b0 0x7f198c0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f198b7fe640 1 -- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c0b55a0 msgr2=0x7f198c0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f198b7fe640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c0b55a0 0x7f198c0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f198b7fe640 1 -- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f198c0d01b0 msgr2=0x7f198c0cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f198b7fe640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f198c0d01b0 0x7f198c0cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f198b7fe640 1 -- 172.21.15.98:0/2223204730 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c0b3fe0 con 0x7f198c161560 2024-09-10T21:00:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f1983fff640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f198c0d01b0 0x7f198c0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:21.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.867+0000 7f198b7fe640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198c161560 0x7f198c1f79d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f198400c7a0 tx=0x7f198400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:21.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.868+0000 7f19837fe640 1 -- 172.21.15.98:0/2223204730 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984003b70 con 0x7f198c161560 2024-09-10T21:00:21.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.868+0000 7f19837fe640 1 -- 172.21.15.98:0/2223204730 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1984003d10 con 0x7f198c161560 2024-09-10T21:00:21.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.868+0000 7f19917ec640 1 -- 172.21.15.98:0/2223204730 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198c0d05e0 con 0x7f198c161560 2024-09-10T21:00:21.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.868+0000 7f19837fe640 1 -- 172.21.15.98:0/2223204730 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198400aab0 con 0x7f198c161560 2024-09-10T21:00:21.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.868+0000 7f19917ec640 1 -- 172.21.15.98:0/2223204730 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f198c09dc50 con 0x7f198c161560 2024-09-10T21:00:21.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.869+0000 7f19917ec640 1 -- 172.21.15.98:0/2223204730 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198c0cf500 con 0x7f198c161560 2024-09-10T21:00:21.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.869+0000 7f19837fe640 1 -- 172.21.15.98:0/2223204730 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f198400ac50 con 0x7f198c161560 2024-09-10T21:00:21.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.870+0000 7f19837fe640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f196c077f70 0x7f196c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:21.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.870+0000 7f19837fe640 1 -- 172.21.15.98:0/2223204730 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f1984099a30 con 0x7f198c161560 2024-09-10T21:00:21.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.870+0000 7f198affd640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f196c077f70 0x7f196c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:21.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.871+0000 7f198affd640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f196c077f70 0x7f196c07a430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f197c033000 tx=0x7f197c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:21.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:21.874+0000 7f19837fe640 1 -- 172.21.15.98:0/2223204730 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f19840626e0 con 0x7f198c161560 2024-09-10T21:00:22.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:22.118+0000 7f19917ec640 1 -- 172.21.15.98:0/2223204730 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]} v 0) -- 0x7f198c0b47d0 con 0x7f198c161560 2024-09-10T21:00:22.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:22.441+0000 7f19837fe640 1 -- 172.21.15.98:0/2223204730 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cap_flush.TestCapFlush.test_replay_create"]}]=0 v108) ==== 138+0+0 (secure 0 0 0) 0x7f1984061fe0 con 0x7f198c161560 2024-09-10T21:00:22.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:22.444+0000 7f19917ec640 1 -- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f196c077f70 msgr2=0x7f196c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:22.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:22.444+0000 7f19917ec640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f196c077f70 0x7f196c07a430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f197c033000 tx=0x7f197c004730 comp rx=0 tx=0).stop 2024-09-10T21:00:22.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:22.444+0000 7f19917ec640 1 -- 172.21.15.98:0/2223204730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198c161560 msgr2=0x7f198c1f79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:22.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:22.444+0000 7f19917ec640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198c161560 0x7f198c1f79d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f198400c7a0 tx=0x7f198400cc60 comp rx=0 tx=0).stop 2024-09-10T21:00:22.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:22.444+0000 7f19917ec640 1 -- 172.21.15.98:0/2223204730 shutdown_connections 2024-09-10T21:00:22.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:22.444+0000 7f19917ec640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f196c077f70 0x7f196c07a430 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:22.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:22.444+0000 7f19917ec640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198c161560 0x7f198c1f79d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:22.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:22.444+0000 7f19917ec640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f198c0d01b0 0x7f198c0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:22.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:22.444+0000 7f19917ec640 1 --2- 172.21.15.98:0/2223204730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f198c0b55a0 0x7f198c0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:22.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:22.444+0000 7f19917ec640 1 -- 172.21.15.98:0/2223204730 >> 172.21.15.98:0/2223204730 conn(0x7f198c09d170 msgr2=0x7f198c0b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:22.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:22.444+0000 7f19917ec640 1 -- 172.21.15.98:0/2223204730 shutdown_connections 2024-09-10T21:00:22.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:22.444+0000 7f19917ec640 1 -- 172.21.15.98:0/2223204730 wait complete. 2024-09-10T21:00:22.459 INFO:tasks.cephfs_test_runner:MDS starts to handle client caps when it enters clientreplay stage. ... ok 2024-09-10T21:00:22.459 INFO:tasks.cephfs_test_runner: 2024-09-10T21:00:22.460 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-10T21:00:22.460 INFO:tasks.cephfs_test_runner:Ran 1 test in 60.285s 2024-09-10T21:00:22.460 INFO:tasks.cephfs_test_runner: 2024-09-10T21:00:22.460 INFO:tasks.cephfs_test_runner:OK 2024-09-10T21:00:22.460 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-10T21:00:22.468 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T21:00:22.517 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T21:00:22.517 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T21:00:22.517 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:00:22.533 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056024,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2547 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:00:22.533 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-10T21:00:22.534 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:00:22.590 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056024,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2547 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:00:22.590 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T21:00:22.590 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:00:22.610 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=154 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:00:22.610 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-09-10T21:00:22.610 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:00:22.668 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=154 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:00:22.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:22.669 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-10T21:00:22.685 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:00:22.685 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:00:22.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:22.686 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:00:22.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:22.752 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:00:22.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:22.819 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link delete ceph-brx 2024-09-10T21:00:22.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:22.911 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-10T21:00:22.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:22.926 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link delete ceph-brx 2024-09-10T21:00:22.995 INFO:teuthology.orchestra.run.smithi098.stderr:Cannot find device "ceph-brx" 2024-09-10T21:00:22.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:00:22.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:22.997 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-09-10T21:00:23.016 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T21:00:23.016 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:00:23.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:23.017 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:00:23.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:23.086 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:00:23.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:23.155 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-09-10T21:00:23.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:23.254 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-09-10T21:00:23.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:00:23.271 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-09-10T21:00:23.338 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2024-09-10T21:00:23.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:00:23.341 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T21:00:23.354 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:00:23.354 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:00:23.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.453+0000 7f6cad633640 1 -- 172.21.15.98:0/2036678068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ca8072680 msgr2=0x7f6ca80ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:23.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.453+0000 7f6cad633640 1 --2- 172.21.15.98:0/2036678068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ca8072680 0x7f6ca80ceb00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009830 tx=0x7f6c9002a480 comp rx=0 tx=0).stop 2024-09-10T21:00:23.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.453+0000 7f6cad633640 1 -- 172.21.15.98:0/2036678068 shutdown_connections 2024-09-10T21:00:23.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.453+0000 7f6cad633640 1 --2- 172.21.15.98:0/2036678068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ca8072ad0 0x7f6ca80cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.453+0000 7f6cad633640 1 --2- 172.21.15.98:0/2036678068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ca8072680 0x7f6ca80ceb00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.453+0000 7f6cad633640 1 -- 172.21.15.98:0/2036678068 >> 172.21.15.98:0/2036678068 conn(0x7f6ca80b1760 msgr2=0x7f6ca80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:23.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.453+0000 7f6cad633640 1 -- 172.21.15.98:0/2036678068 shutdown_connections 2024-09-10T21:00:23.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.453+0000 7f6cad633640 1 -- 172.21.15.98:0/2036678068 wait complete. 2024-09-10T21:00:23.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.454+0000 7f6cad633640 1 Processor -- start 2024-09-10T21:00:23.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.454+0000 7f6cad633640 1 -- start start 2024-09-10T21:00:23.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6cad633640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca8072680 0x7f6ca80aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:23.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6cad633640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ca8072ad0 0x7f6ca815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6cad633640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ca8144780 0x7f6ca815b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6cad633640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca815c9f0 con 0x7f6ca8072ad0 2024-09-10T21:00:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6cad633640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca815c870 con 0x7f6ca8072680 2024-09-10T21:00:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6cad633640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca815cb70 con 0x7f6ca8144780 2024-09-10T21:00:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6c9e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ca8072ad0 0x7f6ca815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6c9e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ca8072ad0 0x7f6ca815b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41914/0 (socket says 172.21.15.98:41914) 2024-09-10T21:00:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6c9e7fc640 1 -- 172.21.15.98:0/2721907935 learned_addr learned my addr 172.21.15.98:0/2721907935 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6ca77fe640 1 --2- 172.21.15.98:0/2721907935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ca8144780 0x7f6ca815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6c9e7fc640 1 -- 172.21.15.98:0/2721907935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ca8144780 msgr2=0x7f6ca815b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:23.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6c9e7fc640 1 --2- 172.21.15.98:0/2721907935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ca8144780 0x7f6ca815b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6c9e7fc640 1 -- 172.21.15.98:0/2721907935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca8072680 msgr2=0x7f6ca80aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:23.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6c9e7fc640 1 --2- 172.21.15.98:0/2721907935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca8072680 0x7f6ca80aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6c9e7fc640 1 -- 172.21.15.98:0/2721907935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca81f61c0 con 0x7f6ca8072ad0 2024-09-10T21:00:23.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.455+0000 7f6c9e7fc640 1 --2- 172.21.15.98:0/2721907935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ca8072ad0 0x7f6ca815b1f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6c90000c00 tx=0x7f6c90032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:23.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.456+0000 7f6c9ffff640 1 -- 172.21.15.98:0/2721907935 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90040070 con 0x7f6ca8072ad0 2024-09-10T21:00:23.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.456+0000 7f6c9ffff640 1 -- 172.21.15.98:0/2721907935 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c900038e0 con 0x7f6ca8072ad0 2024-09-10T21:00:23.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.456+0000 7f6cad633640 1 -- 172.21.15.98:0/2721907935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca81f64a0 con 0x7f6ca8072ad0 2024-09-10T21:00:23.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.456+0000 7f6c9ffff640 1 -- 172.21.15.98:0/2721907935 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90003d30 con 0x7f6ca8072ad0 2024-09-10T21:00:23.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.456+0000 7f6cad633640 1 -- 172.21.15.98:0/2721907935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca81f6a60 con 0x7f6ca8072ad0 2024-09-10T21:00:23.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.457+0000 7f6cad633640 1 -- 172.21.15.98:0/2721907935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca815d020 con 0x7f6ca8072ad0 2024-09-10T21:00:23.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.457+0000 7f6c9ffff640 1 -- 172.21.15.98:0/2721907935 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6c9004d030 con 0x7f6ca8072ad0 2024-09-10T21:00:23.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.458+0000 7f6c9ffff640 1 --2- 172.21.15.98:0/2721907935 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6c88077f70 0x7f6c8807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:23.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.458+0000 7f6c9ffff640 1 -- 172.21.15.98:0/2721907935 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f6c900bb260 con 0x7f6ca8072ad0 2024-09-10T21:00:23.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.458+0000 7f6ca6ffd640 1 --2- 172.21.15.98:0/2721907935 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6c88077f70 0x7f6c8807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:23.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.458+0000 7f6ca6ffd640 1 --2- 172.21.15.98:0/2721907935 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6c88077f70 0x7f6c8807a430 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6c980046b0 tx=0x7f6c98007a70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:23.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.462+0000 7f6c9ffff640 1 -- 172.21.15.98:0/2721907935 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6c90083f10 con 0x7f6ca8072ad0 2024-09-10T21:00:23.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.633+0000 7f6cad633640 1 -- 172.21.15.98:0/2721907935 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ca80b4d40 con 0x7f6c88077f70 2024-09-10T21:00:23.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.634+0000 7f6c9ffff640 1 -- 172.21.15.98:0/2721907935 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13405 (secure 0 0 0) 0x7f6ca80b4d40 con 0x7f6c88077f70 2024-09-10T21:00:23.638 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:23.639 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-10T21:00:23.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.636+0000 7f6cad633640 1 -- 172.21.15.98:0/2721907935 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6c88077f70 msgr2=0x7f6c8807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:23.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.636+0000 7f6cad633640 1 --2- 172.21.15.98:0/2721907935 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6c88077f70 0x7f6c8807a430 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6c980046b0 tx=0x7f6c98007a70 comp rx=0 tx=0).stop 2024-09-10T21:00:23.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.636+0000 7f6cad633640 1 -- 172.21.15.98:0/2721907935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ca8072ad0 msgr2=0x7f6ca815b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:23.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.636+0000 7f6cad633640 1 --2- 172.21.15.98:0/2721907935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ca8072ad0 0x7f6ca815b1f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6c90000c00 tx=0x7f6c90032c10 comp rx=0 tx=0).stop 2024-09-10T21:00:23.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.636+0000 7f6cad633640 1 -- 172.21.15.98:0/2721907935 shutdown_connections 2024-09-10T21:00:23.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.636+0000 7f6cad633640 1 --2- 172.21.15.98:0/2721907935 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6c88077f70 0x7f6c8807a430 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.636+0000 7f6cad633640 1 --2- 172.21.15.98:0/2721907935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ca8144780 0x7f6ca815b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.636+0000 7f6cad633640 1 --2- 172.21.15.98:0/2721907935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ca8072ad0 0x7f6ca815b1f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.636+0000 7f6cad633640 1 --2- 172.21.15.98:0/2721907935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca8072680 0x7f6ca80aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.636+0000 7f6cad633640 1 -- 172.21.15.98:0/2721907935 >> 172.21.15.98:0/2721907935 conn(0x7f6ca80b1760 msgr2=0x7f6ca80b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:23.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.636+0000 7f6cad633640 1 -- 172.21.15.98:0/2721907935 shutdown_connections 2024-09-10T21:00:23.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.636+0000 7f6cad633640 1 -- 172.21.15.98:0/2721907935 wait complete. 2024-09-10T21:00:23.651 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-09-10T21:00:22.930716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221216,"kb_used_data":4992,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753504,"statfs":{"total":773094113280,"available":772867588096,"internally_reserved":0,"allocated":5111808,"data_stored":2902404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16341,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-74,"num_write_kb":-42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001591"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":152,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T21:00:20.799845+0000","last_change":"2024-09-10T20:58:36.895993+0000","last_active":"2024-09-10T21:00:20.799845+0000","last_peered":"2024-09-10T21:00:20.799845+0000","last_clean":"2024-09-10T21:00:20.799845+0000","last_became_active":"2024-09-10T20:58:36.894988+0000","last_became_peered":"2024-09-10T20:58:36.894988+0000","last_unstale":"2024-09-10T21:00:20.799845+0000","last_undegraded":"2024-09-10T21:00:20.799845+0000","last_fullsized":"2024-09-10T21:00:20.799845+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_clean_scrub_stamp":"2024-09-10T20:58:35.588702+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.082528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1101824,"data_stored":817222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":211666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":211688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":507904,"data_stored":226818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542168,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":211154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542168,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":211174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1077248,"data_stored":801486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":211196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:00:23.652 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:00:23.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.749+0000 7f45f297d640 1 -- 172.21.15.98:0/3377320145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45ec0cf110 msgr2=0x7f45ec0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:23.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.749+0000 7f45f297d640 1 --2- 172.21.15.98:0/3377320145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45ec0cf110 0x7f45ec0b47b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f45e4005fd0 tx=0x7f45e402a8b0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.749+0000 7f45f297d640 1 -- 172.21.15.98:0/3377320145 shutdown_connections 2024-09-10T21:00:23.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.749+0000 7f45f297d640 1 --2- 172.21.15.98:0/3377320145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45ec0cf110 0x7f45ec0b47b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.749+0000 7f45f297d640 1 --2- 172.21.15.98:0/3377320145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45ec06e2e0 0x7f45ec15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.749+0000 7f45f297d640 1 -- 172.21.15.98:0/3377320145 >> 172.21.15.98:0/3377320145 conn(0x7f45ec0b1760 msgr2=0x7f45ec0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:23.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.749+0000 7f45f297d640 1 -- 172.21.15.98:0/3377320145 shutdown_connections 2024-09-10T21:00:23.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.749+0000 7f45f297d640 1 -- 172.21.15.98:0/3377320145 wait complete. 2024-09-10T21:00:23.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.750+0000 7f45f297d640 1 Processor -- start 2024-09-10T21:00:23.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.750+0000 7f45f297d640 1 -- start start 2024-09-10T21:00:23.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f297d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45ec06e2e0 0x7f45ec142c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:23.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f297d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45ec0cf110 0x7f45ec0cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:23.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f297d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45ec15e150 0x7f45ec1434d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:23.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f297d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec0cc570 con 0x7f45ec15e150 2024-09-10T21:00:23.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f297d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec0729d0 con 0x7f45ec06e2e0 2024-09-10T21:00:23.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f297d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f45ec0cc6f0 con 0x7f45ec0cf110 2024-09-10T21:00:23.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f0f1a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45ec15e150 0x7f45ec1434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:23.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f0f1a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45ec15e150 0x7f45ec1434d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41920/0 (socket says 172.21.15.98:41920) 2024-09-10T21:00:23.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f0f1a640 1 -- 172.21.15.98:0/1154244235 learned_addr learned my addr 172.21.15.98:0/1154244235 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:23.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45eb7fe640 1 --2- 172.21.15.98:0/1154244235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45ec0cf110 0x7f45ec0cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:23.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45ebfff640 1 --2- 172.21.15.98:0/1154244235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45ec06e2e0 0x7f45ec142c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:23.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f0f1a640 1 -- 172.21.15.98:0/1154244235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45ec0cf110 msgr2=0x7f45ec0cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:23.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f0f1a640 1 --2- 172.21.15.98:0/1154244235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45ec0cf110 0x7f45ec0cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f0f1a640 1 -- 172.21.15.98:0/1154244235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45ec06e2e0 msgr2=0x7f45ec142c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:23.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f0f1a640 1 --2- 172.21.15.98:0/1154244235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45ec06e2e0 0x7f45ec142c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f0f1a640 1 -- 172.21.15.98:0/1154244235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ec154520 con 0x7f45ec15e150 2024-09-10T21:00:23.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.751+0000 7f45f0f1a640 1 --2- 172.21.15.98:0/1154244235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45ec15e150 0x7f45ec1434d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009cd0 tx=0x7f45e4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:23.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.752+0000 7f45e8ff9640 1 -- 172.21.15.98:0/1154244235 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4040070 con 0x7f45ec15e150 2024-09-10T21:00:23.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.752+0000 7f45e8ff9640 1 -- 172.21.15.98:0/1154244235 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45e4004130 con 0x7f45ec15e150 2024-09-10T21:00:23.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.752+0000 7f45f297d640 1 -- 172.21.15.98:0/1154244235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ec154800 con 0x7f45ec15e150 2024-09-10T21:00:23.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.752+0000 7f45e8ff9640 1 -- 172.21.15.98:0/1154244235 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4004540 con 0x7f45ec15e150 2024-09-10T21:00:23.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.752+0000 7f45f297d640 1 -- 172.21.15.98:0/1154244235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ec144210 con 0x7f45ec15e150 2024-09-10T21:00:23.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.753+0000 7f45f297d640 1 -- 172.21.15.98:0/1154244235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45ec0ccba0 con 0x7f45ec15e150 2024-09-10T21:00:23.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.753+0000 7f45e8ff9640 1 -- 172.21.15.98:0/1154244235 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f45e404d030 con 0x7f45ec15e150 2024-09-10T21:00:23.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.754+0000 7f45e8ff9640 1 --2- 172.21.15.98:0/1154244235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f45d8077fa0 0x7f45d807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:23.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.754+0000 7f45e8ff9640 1 -- 172.21.15.98:0/1154244235 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f45e40bb070 con 0x7f45ec15e150 2024-09-10T21:00:23.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.754+0000 7f45ebfff640 1 --2- 172.21.15.98:0/1154244235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f45d8077fa0 0x7f45d807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:23.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.754+0000 7f45ebfff640 1 --2- 172.21.15.98:0/1154244235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f45d8077fa0 0x7f45d807a460 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f45ec14a1d0 tx=0x7f45dc007e90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:23.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.758+0000 7f45e8ff9640 1 -- 172.21.15.98:0/1154244235 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f45e40836d0 con 0x7f45ec15e150 2024-09-10T21:00:23.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.931+0000 7f45f297d640 1 -- 172.21.15.98:0/1154244235 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f45ec146eb0 con 0x7f45d8077fa0 2024-09-10T21:00:23.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.932+0000 7f45e8ff9640 1 -- 172.21.15.98:0/1154244235 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13405 (secure 0 0 0) 0x7f45ec146eb0 con 0x7f45d8077fa0 2024-09-10T21:00:23.937 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:23.937 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-10T21:00:23.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.934+0000 7f45f297d640 1 -- 172.21.15.98:0/1154244235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f45d8077fa0 msgr2=0x7f45d807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:23.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.934+0000 7f45f297d640 1 --2- 172.21.15.98:0/1154244235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f45d8077fa0 0x7f45d807a460 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f45ec14a1d0 tx=0x7f45dc007e90 comp rx=0 tx=0).stop 2024-09-10T21:00:23.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.934+0000 7f45f297d640 1 -- 172.21.15.98:0/1154244235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45ec15e150 msgr2=0x7f45ec1434d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:23.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.934+0000 7f45f297d640 1 --2- 172.21.15.98:0/1154244235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45ec15e150 0x7f45ec1434d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009cd0 tx=0x7f45e4003970 comp rx=0 tx=0).stop 2024-09-10T21:00:23.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.934+0000 7f45f297d640 1 -- 172.21.15.98:0/1154244235 shutdown_connections 2024-09-10T21:00:23.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.934+0000 7f45f297d640 1 --2- 172.21.15.98:0/1154244235 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f45d8077fa0 0x7f45d807a460 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.934+0000 7f45f297d640 1 --2- 172.21.15.98:0/1154244235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45ec15e150 0x7f45ec1434d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.934+0000 7f45f297d640 1 --2- 172.21.15.98:0/1154244235 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45ec0cf110 0x7f45ec0cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.934+0000 7f45f297d640 1 --2- 172.21.15.98:0/1154244235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45ec06e2e0 0x7f45ec142c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:23.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.934+0000 7f45f297d640 1 -- 172.21.15.98:0/1154244235 >> 172.21.15.98:0/1154244235 conn(0x7f45ec0b1760 msgr2=0x7f45ec06f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:23.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.934+0000 7f45f297d640 1 -- 172.21.15.98:0/1154244235 shutdown_connections 2024-09-10T21:00:23.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:23.934+0000 7f45f297d640 1 -- 172.21.15.98:0/1154244235 wait complete. 2024-09-10T21:00:23.949 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-09-10T21:00:22.930716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221216,"kb_used_data":4992,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753504,"statfs":{"total":773094113280,"available":772867588096,"internally_reserved":0,"allocated":5111808,"data_stored":2902404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16341,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-74,"num_write_kb":-42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001591"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":152,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T21:00:20.799845+0000","last_change":"2024-09-10T20:58:36.895993+0000","last_active":"2024-09-10T21:00:20.799845+0000","last_peered":"2024-09-10T21:00:20.799845+0000","last_clean":"2024-09-10T21:00:20.799845+0000","last_became_active":"2024-09-10T20:58:36.894988+0000","last_became_peered":"2024-09-10T20:58:36.894988+0000","last_unstale":"2024-09-10T21:00:20.799845+0000","last_undegraded":"2024-09-10T21:00:20.799845+0000","last_fullsized":"2024-09-10T21:00:20.799845+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_clean_scrub_stamp":"2024-09-10T20:58:35.588702+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.082528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1101824,"data_stored":817222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":211666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":211688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":507904,"data_stored":226818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542168,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":211154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542168,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":211174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1077248,"data_stored":801486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":211196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:00:23.950 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:00:23.950 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:00:24.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.048+0000 7f2c4ea16640 1 -- 172.21.15.98:0/302053449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c30004800 msgr2=0x7f2c30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:24.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.048+0000 7f2c4ea16640 1 --2- 172.21.15.98:0/302053449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c30004800 0x7f2c30004c80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c001690 tx=0x7f2c30005830 comp rx=0 tx=0).stop 2024-09-10T21:00:24.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.048+0000 7f2c4ea16640 1 -- 172.21.15.98:0/302053449 shutdown_connections 2024-09-10T21:00:24.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.048+0000 7f2c4ea16640 1 --2- 172.21.15.98:0/302053449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c30004800 0x7f2c30004c80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.048+0000 7f2c4ea16640 1 -- 172.21.15.98:0/302053449 >> 172.21.15.98:0/302053449 conn(0x7f2c480b1760 msgr2=0x7f2c480b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:24.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.048+0000 7f2c4ea16640 1 -- 172.21.15.98:0/302053449 shutdown_connections 2024-09-10T21:00:24.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.048+0000 7f2c4ea16640 1 -- 172.21.15.98:0/302053449 wait complete. 2024-09-10T21:00:24.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.049+0000 7f2c4ea16640 1 Processor -- start 2024-09-10T21:00:24.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.049+0000 7f2c4ea16640 1 -- start start 2024-09-10T21:00:24.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c4ea16640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c30004800 0x7f2c480a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:24.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c4ea16640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c480aa3d0 0x7f2c481428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:24.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c4ea16640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c4815a7e0 0x7f2c48142e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:24.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c4ea16640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c480cec80 con 0x7f2c30004800 2024-09-10T21:00:24.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c4ea16640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2c480ceb00 con 0x7f2c4815a7e0 2024-09-10T21:00:24.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c4ea16640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2c480cee00 con 0x7f2c480aa3d0 2024-09-10T21:00:24.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c47fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c30004800 0x7f2c480a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:24.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c47fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c30004800 0x7f2c480a9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41940/0 (socket says 172.21.15.98:41940) 2024-09-10T21:00:24.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c47fff640 1 -- 172.21.15.98:0/1780042438 learned_addr learned my addr 172.21.15.98:0/1780042438 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:24.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c477fe640 1 --2- 172.21.15.98:0/1780042438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c480aa3d0 0x7f2c481428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:24.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c4cfb3640 1 --2- 172.21.15.98:0/1780042438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c4815a7e0 0x7f2c48142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:24.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c47fff640 1 -- 172.21.15.98:0/1780042438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c480aa3d0 msgr2=0x7f2c481428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:24.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c47fff640 1 --2- 172.21.15.98:0/1780042438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c480aa3d0 0x7f2c481428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c47fff640 1 -- 172.21.15.98:0/1780042438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c4815a7e0 msgr2=0x7f2c48142e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:24.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c47fff640 1 --2- 172.21.15.98:0/1780042438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c4815a7e0 0x7f2c48142e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c47fff640 1 -- 172.21.15.98:0/1780042438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4813c2f0 con 0x7f2c30004800 2024-09-10T21:00:24.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c47fff640 1 --2- 172.21.15.98:0/1780042438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c30004800 0x7f2c480a9e90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2c38011ae0 tx=0x7f2c38009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:24.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c44ff9640 1 -- 172.21.15.98:0/1780042438 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3801e070 con 0x7f2c30004800 2024-09-10T21:00:24.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c44ff9640 1 -- 172.21.15.98:0/1780042438 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c3800fc10 con 0x7f2c30004800 2024-09-10T21:00:24.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c4ea16640 1 -- 172.21.15.98:0/1780042438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4813c590 con 0x7f2c30004800 2024-09-10T21:00:24.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.050+0000 7f2c44ff9640 1 -- 172.21.15.98:0/1780042438 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38019410 con 0x7f2c30004800 2024-09-10T21:00:24.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.051+0000 7f2c4ea16640 1 -- 172.21.15.98:0/1780042438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c480aa930 con 0x7f2c30004800 2024-09-10T21:00:24.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.051+0000 7f2c4ea16640 1 -- 172.21.15.98:0/1780042438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c480b3f10 con 0x7f2c30004800 2024-09-10T21:00:24.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.052+0000 7f2c44ff9640 1 -- 172.21.15.98:0/1780042438 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2c380195b0 con 0x7f2c30004800 2024-09-10T21:00:24.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.052+0000 7f2c44ff9640 1 --2- 172.21.15.98:0/1780042438 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2c34077fa0 0x7f2c3407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:24.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.053+0000 7f2c44ff9640 1 -- 172.21.15.98:0/1780042438 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f2c3809a600 con 0x7f2c30004800 2024-09-10T21:00:24.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.053+0000 7f2c477fe640 1 --2- 172.21.15.98:0/1780042438 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2c34077fa0 0x7f2c3407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:24.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.053+0000 7f2c477fe640 1 --2- 172.21.15.98:0/1780042438 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2c34077fa0 0x7f2c3407a460 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c032000 tx=0x7f2c3c001190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:24.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.057+0000 7f2c44ff9640 1 -- 172.21.15.98:0/1780042438 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2c38062c60 con 0x7f2c30004800 2024-09-10T21:00:24.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.228+0000 7f2c4ea16640 1 -- 172.21.15.98:0/1780042438 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c480cf810 con 0x7f2c34077fa0 2024-09-10T21:00:24.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.228+0000 7f2c44ff9640 1 -- 172.21.15.98:0/1780042438 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13405 (secure 0 0 0) 0x7f2c480cf810 con 0x7f2c34077fa0 2024-09-10T21:00:24.233 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:24.233 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-10T21:00:24.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.231+0000 7f2c4ea16640 1 -- 172.21.15.98:0/1780042438 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2c34077fa0 msgr2=0x7f2c3407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:24.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.231+0000 7f2c4ea16640 1 --2- 172.21.15.98:0/1780042438 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2c34077fa0 0x7f2c3407a460 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c032000 tx=0x7f2c3c001190 comp rx=0 tx=0).stop 2024-09-10T21:00:24.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.231+0000 7f2c4ea16640 1 -- 172.21.15.98:0/1780042438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c30004800 msgr2=0x7f2c480a9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:24.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.231+0000 7f2c4ea16640 1 --2- 172.21.15.98:0/1780042438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c30004800 0x7f2c480a9e90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2c38011ae0 tx=0x7f2c38009eb0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.231+0000 7f2c4ea16640 1 -- 172.21.15.98:0/1780042438 shutdown_connections 2024-09-10T21:00:24.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.231+0000 7f2c4ea16640 1 --2- 172.21.15.98:0/1780042438 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2c34077fa0 0x7f2c3407a460 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.231+0000 7f2c4ea16640 1 --2- 172.21.15.98:0/1780042438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c4815a7e0 0x7f2c48142e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.231+0000 7f2c4ea16640 1 --2- 172.21.15.98:0/1780042438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c480aa3d0 0x7f2c481428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.231+0000 7f2c4ea16640 1 --2- 172.21.15.98:0/1780042438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c30004800 0x7f2c480a9e90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.231+0000 7f2c4ea16640 1 -- 172.21.15.98:0/1780042438 >> 172.21.15.98:0/1780042438 conn(0x7f2c480b1760 msgr2=0x7f2c4815d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:24.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.231+0000 7f2c4ea16640 1 -- 172.21.15.98:0/1780042438 shutdown_connections 2024-09-10T21:00:24.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.231+0000 7f2c4ea16640 1 -- 172.21.15.98:0/1780042438 wait complete. 2024-09-10T21:00:24.246 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-09-10T21:00:22.930716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221216,"kb_used_data":4992,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753504,"statfs":{"total":773094113280,"available":772867588096,"internally_reserved":0,"allocated":5111808,"data_stored":2902404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16341,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-86,"num_read_kb":-88,"num_write":-74,"num_write_kb":-42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001591"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":152,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-10T21:00:20.799845+0000","last_change":"2024-09-10T20:58:36.895993+0000","last_active":"2024-09-10T21:00:20.799845+0000","last_peered":"2024-09-10T21:00:20.799845+0000","last_clean":"2024-09-10T21:00:20.799845+0000","last_became_active":"2024-09-10T20:58:36.894988+0000","last_became_peered":"2024-09-10T20:58:36.894988+0000","last_unstale":"2024-09-10T21:00:20.799845+0000","last_undegraded":"2024-09-10T21:00:20.799845+0000","last_fullsized":"2024-09-10T21:00:20.799845+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_clean_scrub_stamp":"2024-09-10T20:58:35.588702+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.082528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1101824,"data_stored":817222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":211666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":211688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":507904,"data_stored":226818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542168,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":211154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542168,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":211174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1077248,"data_stored":801486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":487424,"data_stored":211196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:00:24.246 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:00:24.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.344+0000 7f2918e2b640 1 -- 172.21.15.98:0/348633333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29140b7300 msgr2=0x7f29140d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:24.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.344+0000 7f2918e2b640 1 --2- 172.21.15.98:0/348633333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29140b7300 0x7f29140d24e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f28fc005fd0 tx=0x7f28fc02a4a0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.344+0000 7f2918e2b640 1 -- 172.21.15.98:0/348633333 shutdown_connections 2024-09-10T21:00:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.344+0000 7f2918e2b640 1 --2- 172.21.15.98:0/348633333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29140b7300 0x7f29140d24e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.344+0000 7f2918e2b640 1 --2- 172.21.15.98:0/348633333 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f291415e060 0x7f29140b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.344+0000 7f2918e2b640 1 -- 172.21.15.98:0/348633333 >> 172.21.15.98:0/348633333 conn(0x7f29140d3f70 msgr2=0x7f29140d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.344+0000 7f2918e2b640 1 -- 172.21.15.98:0/348633333 shutdown_connections 2024-09-10T21:00:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.344+0000 7f2918e2b640 1 -- 172.21.15.98:0/348633333 wait complete. 2024-09-10T21:00:24.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.345+0000 7f2918e2b640 1 Processor -- start 2024-09-10T21:00:24.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.345+0000 7f2918e2b640 1 -- start start 2024-09-10T21:00:24.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f2918e2b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29140d4af0 0x7f29141f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:24.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f2918e2b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f291415e060 0x7f29141f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:24.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f2918e2b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29140ccee0 0x7f291415fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:24.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f2918e2b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f29140d2db0 con 0x7f29140d4af0 2024-09-10T21:00:24.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f2918e2b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f29140d2c30 con 0x7f291415e060 2024-09-10T21:00:24.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f2918e2b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f29140d2f30 con 0x7f29140ccee0 2024-09-10T21:00:24.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f291259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29140d4af0 0x7f29141f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:24.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f291259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29140d4af0 0x7f29141f4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41970/0 (socket says 172.21.15.98:41970) 2024-09-10T21:00:24.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f291259c640 1 -- 172.21.15.98:0/3633408158 learned_addr learned my addr 172.21.15.98:0/3633408158 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:24.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f2912d9d640 1 --2- 172.21.15.98:0/3633408158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29140ccee0 0x7f291415fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:24.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f291259c640 1 -- 172.21.15.98:0/3633408158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29140ccee0 msgr2=0x7f291415fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:24.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f291259c640 1 --2- 172.21.15.98:0/3633408158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29140ccee0 0x7f291415fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f290bfff640 1 --2- 172.21.15.98:0/3633408158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f291415e060 0x7f29141f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:24.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f291259c640 1 -- 172.21.15.98:0/3633408158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f291415e060 msgr2=0x7f29141f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:24.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f291259c640 1 --2- 172.21.15.98:0/3633408158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f291415e060 0x7f29141f53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.346+0000 7f291259c640 1 -- 172.21.15.98:0/3633408158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29141f5930 con 0x7f29140d4af0 2024-09-10T21:00:24.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.347+0000 7f291259c640 1 --2- 172.21.15.98:0/3633408158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29140d4af0 0x7f29141f4970 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f290400d700 tx=0x7f290400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:24.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.347+0000 7f290affd640 1 -- 172.21.15.98:0/3633408158 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904017070 con 0x7f29140d4af0 2024-09-10T21:00:24.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.347+0000 7f290affd640 1 -- 172.21.15.98:0/3633408158 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2904005310 con 0x7f29140d4af0 2024-09-10T21:00:24.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.347+0000 7f290affd640 1 -- 172.21.15.98:0/3633408158 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2904009820 con 0x7f29140d4af0 2024-09-10T21:00:24.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.347+0000 7f2918e2b640 1 -- 172.21.15.98:0/3633408158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291415e940 con 0x7f29140d4af0 2024-09-10T21:00:24.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.347+0000 7f2918e2b640 1 -- 172.21.15.98:0/3633408158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291415ef00 con 0x7f29140d4af0 2024-09-10T21:00:24.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.348+0000 7f2918e2b640 1 -- 172.21.15.98:0/3633408158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29140d24e0 con 0x7f29140d4af0 2024-09-10T21:00:24.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.348+0000 7f290affd640 1 -- 172.21.15.98:0/3633408158 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f29040099c0 con 0x7f29140d4af0 2024-09-10T21:00:24.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.349+0000 7f290affd640 1 --2- 172.21.15.98:0/3633408158 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f28f4077f70 0x7f28f407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:24.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.349+0000 7f290affd640 1 -- 172.21.15.98:0/3633408158 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f2904099d50 con 0x7f29140d4af0 2024-09-10T21:00:24.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.349+0000 7f290bfff640 1 --2- 172.21.15.98:0/3633408158 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f28f4077f70 0x7f28f407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:24.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.350+0000 7f290bfff640 1 --2- 172.21.15.98:0/3633408158 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f28f4077f70 0x7f28f407a430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f28fc033000 tx=0x7f28fc004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:24.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.357+0000 7f290affd640 1 -- 172.21.15.98:0/3633408158 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f290409d030 con 0x7f29140d4af0 2024-09-10T21:00:24.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.538+0000 7f2918e2b640 1 -- 172.21.15.98:0/3633408158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f291408dfa0 con 0x7f29140d4af0 2024-09-10T21:00:24.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.539+0000 7f290affd640 1 -- 172.21.15.98:0/3633408158 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+13449 (secure 0 0 0) 0x7f2904062970 con 0x7f29140d4af0 2024-09-10T21:00:24.543 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:24.543 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":44,"fsid":"41041377-a016-49fd-be1f-323d0b834557","created":"2024-09-10T20:58:22.543787+0000","modified":"2024-09-10T21:00:21.436710+0000","last_up_change":"2024-09-10T20:58:35.588702+0000","last_in_change":"2024-09-10T20:58:25.547705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:58:34.920029+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c2fd6ed-6940-4fc6-926a-85685cfdadd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6825","nonce":3415066506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6827","nonce":3415066506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6831","nonce":3415066506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":3415066506},{"type":"v1","addr":"172.21.15.98:6829","nonce":3415066506}]},"public_addr":"172.21.15.98:6825/3415066506","cluster_addr":"172.21.15.98:6827/3415066506","heartbeat_back_addr":"172.21.15.98:6831/3415066506","heartbeat_front_addr":"172.21.15.98:6829/3415066506","state":["exists","up"]},{"osd":1,"uuid":"05006fe3-bbb6-4206-916e-43ff3ea7e711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6801","nonce":3091760272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6803","nonce":3091760272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6807","nonce":3091760272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3091760272},{"type":"v1","addr":"172.21.15.98:6805","nonce":3091760272}]},"public_addr":"172.21.15.98:6801/3091760272","cluster_addr":"172.21.15.98:6803/3091760272","heartbeat_back_addr":"172.21.15.98:6807/3091760272","heartbeat_front_addr":"172.21.15.98:6805/3091760272","state":["exists","up"]},{"osd":2,"uuid":"ba3940dc-1a8e-4ca7-a66d-135616696e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6809","nonce":2541446963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6811","nonce":2541446963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6815","nonce":2541446963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2541446963},{"type":"v1","addr":"172.21.15.98:6813","nonce":2541446963}]},"public_addr":"172.21.15.98:6809/2541446963","cluster_addr":"172.21.15.98:6811/2541446963","heartbeat_back_addr":"172.21.15.98:6815/2541446963","heartbeat_front_addr":"172.21.15.98:6813/2541446963","state":["exists","up"]},{"osd":3,"uuid":"686f1fed-2a52-4bcd-b02b-1e682946f04b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6817","nonce":2538250805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6819","nonce":2538250805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6823","nonce":2538250805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":2538250805},{"type":"v1","addr":"172.21.15.98:6821","nonce":2538250805}]},"public_addr":"172.21.15.98:6817/2538250805","cluster_addr":"172.21.15.98:6819/2538250805","heartbeat_back_addr":"172.21.15.98:6823/2538250805","heartbeat_front_addr":"172.21.15.98:6821/2538250805","state":["exists","up"]},{"osd":4,"uuid":"ca6cdff1-7f33-4941-a075-71ff47a38b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6827","nonce":2783440087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6829","nonce":2783440087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6833","nonce":2783440087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2783440087},{"type":"v1","addr":"172.21.15.190:6831","nonce":2783440087}]},"public_addr":"172.21.15.190:6827/2783440087","cluster_addr":"172.21.15.190:6829/2783440087","heartbeat_back_addr":"172.21.15.190:6833/2783440087","heartbeat_front_addr":"172.21.15.190:6831/2783440087","state":["exists","up"]},{"osd":5,"uuid":"914f0013-7767-489d-aeaf-44f67aeb6810","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6803","nonce":904517515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6805","nonce":904517515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6809","nonce":904517515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":904517515},{"type":"v1","addr":"172.21.15.190:6807","nonce":904517515}]},"public_addr":"172.21.15.190:6803/904517515","cluster_addr":"172.21.15.190:6805/904517515","heartbeat_back_addr":"172.21.15.190:6809/904517515","heartbeat_front_addr":"172.21.15.190:6807/904517515","state":["exists","up"]},{"osd":6,"uuid":"fe7b82e6-ec5a-4504-bb5f-7a0c0c5907bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6811","nonce":2106840781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6813","nonce":2106840781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6817","nonce":2106840781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":2106840781},{"type":"v1","addr":"172.21.15.190:6815","nonce":2106840781}]},"public_addr":"172.21.15.190:6811/2106840781","cluster_addr":"172.21.15.190:6813/2106840781","heartbeat_back_addr":"172.21.15.190:6817/2106840781","heartbeat_front_addr":"172.21.15.190:6815/2106840781","state":["exists","up"]},{"osd":7,"uuid":"a4086531-b341-45bb-a07b-a03577a4a0bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6819","nonce":3616946293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6821","nonce":3616946293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6825","nonce":3616946293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3616946293},{"type":"v1","addr":"172.21.15.190:6823","nonce":3616946293}]},"public_addr":"172.21.15.190:6819/3616946293","cluster_addr":"172.21.15.190:6821/3616946293","heartbeat_back_addr":"172.21.15.190:6825/3616946293","heartbeat_front_addr":"172.21.15.190:6823/3616946293","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.640759+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.645513+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.638747+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.077023+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.305746+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:31.768703+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.326609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:32.410372+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6837/3087107175":"2024-09-11T21:00:18.123106+0000","172.21.15.190:6836/3087107175":"2024-09-11T21:00:18.123106+0000","172.21.15.98:6835/4202850803":"2024-09-11T21:00:08.963614+0000","172.21.15.98:6832/4202850803":"2024-09-11T21:00:08.963614+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:00:24.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.541+0000 7f2918e2b640 1 -- 172.21.15.98:0/3633408158 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f28f4077f70 msgr2=0x7f28f407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:24.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.541+0000 7f2918e2b640 1 --2- 172.21.15.98:0/3633408158 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f28f4077f70 0x7f28f407a430 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f28fc033000 tx=0x7f28fc004730 comp rx=0 tx=0).stop 2024-09-10T21:00:24.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.541+0000 7f2918e2b640 1 -- 172.21.15.98:0/3633408158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29140d4af0 msgr2=0x7f29141f4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:24.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.541+0000 7f2918e2b640 1 --2- 172.21.15.98:0/3633408158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29140d4af0 0x7f29141f4970 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f290400d700 tx=0x7f290400dbc0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.541+0000 7f2918e2b640 1 -- 172.21.15.98:0/3633408158 shutdown_connections 2024-09-10T21:00:24.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.541+0000 7f2918e2b640 1 --2- 172.21.15.98:0/3633408158 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f28f4077f70 0x7f28f407a430 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.541+0000 7f2918e2b640 1 --2- 172.21.15.98:0/3633408158 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29140ccee0 0x7f291415fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.541+0000 7f2918e2b640 1 --2- 172.21.15.98:0/3633408158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f291415e060 0x7f29141f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.541+0000 7f2918e2b640 1 --2- 172.21.15.98:0/3633408158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29140d4af0 0x7f29141f4970 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:24.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.541+0000 7f2918e2b640 1 -- 172.21.15.98:0/3633408158 >> 172.21.15.98:0/3633408158 conn(0x7f29140d3f70 msgr2=0x7f29140b78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:24.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.541+0000 7f2918e2b640 1 -- 172.21.15.98:0/3633408158 shutdown_connections 2024-09-10T21:00:24.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:24.541+0000 7f2918e2b640 1 -- 172.21.15.98:0/3633408158 wait complete. 2024-09-10T21:00:25.557 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T21:00:25.557 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:00:25.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.660+0000 7f6a6cc23640 1 -- 172.21.15.98:0/159793288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a6815be00 msgr2=0x7f6a68070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:25.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.660+0000 7f6a6cc23640 1 --2- 172.21.15.98:0/159793288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a6815be00 0x7f6a68070760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6a58005fd0 tx=0x7f6a5802a8b0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.661+0000 7f6a6cc23640 1 -- 172.21.15.98:0/159793288 shutdown_connections 2024-09-10T21:00:25.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.661+0000 7f6a6cc23640 1 --2- 172.21.15.98:0/159793288 >> v2:172.21.15.190:3300/0 conn(0x7f6a680d0a00 0x7f6a680cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.661+0000 7f6a6cc23640 1 --2- 172.21.15.98:0/159793288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a6815be00 0x7f6a68070760 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.661+0000 7f6a6cc23640 1 -- 172.21.15.98:0/159793288 >> 172.21.15.98:0/159793288 conn(0x7f6a680d1d10 msgr2=0x7f6a680d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:25.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.661+0000 7f6a6cc23640 1 -- 172.21.15.98:0/159793288 shutdown_connections 2024-09-10T21:00:25.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.661+0000 7f6a6cc23640 1 -- 172.21.15.98:0/159793288 wait complete. 2024-09-10T21:00:25.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.662+0000 7f6a6cc23640 1 Processor -- start 2024-09-10T21:00:25.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.662+0000 7f6a6cc23640 1 -- start start 2024-09-10T21:00:25.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6cc23640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a680d0a00 0x7f6a68150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:25.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6cc23640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a680d2830 0x7f6a680b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:25.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6cc23640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a6815be00 0x7f6a680b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:25.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6cc23640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a680cd1d0 con 0x7f6a680d0a00 2024-09-10T21:00:25.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6cc23640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6a680cd050 con 0x7f6a680d2830 2024-09-10T21:00:25.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6cc23640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6a680cd350 con 0x7f6a6815be00 2024-09-10T21:00:25.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a680d0a00 0x7f6a68150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:25.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a680d0a00 0x7f6a68150170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41984/0 (socket says 172.21.15.98:41984) 2024-09-10T21:00:25.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6659c640 1 -- 172.21.15.98:0/2379923886 learned_addr learned my addr 172.21.15.98:0/2379923886 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:25.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a66d9d640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a6815be00 0x7f6a680b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:25.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a65d9b640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a680d2830 0x7f6a680b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:25.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6659c640 1 -- 172.21.15.98:0/2379923886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a6815be00 msgr2=0x7f6a680b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:25.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6659c640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a6815be00 0x7f6a680b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6659c640 1 -- 172.21.15.98:0/2379923886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a680d2830 msgr2=0x7f6a680b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:25.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6659c640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a680d2830 0x7f6a680b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6659c640 1 -- 172.21.15.98:0/2379923886 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a680b3e70 con 0x7f6a680d0a00 2024-09-10T21:00:25.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a65d9b640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a680d2830 0x7f6a680b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:25.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a66d9d640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a6815be00 0x7f6a680b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:25.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a6659c640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a680d0a00 0x7f6a68150170 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009cd0 tx=0x7f6a580316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:25.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.663+0000 7f6a5effd640 1 -- 172.21.15.98:0/2379923886 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58031990 con 0x7f6a680d0a00 2024-09-10T21:00:25.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.664+0000 7f6a5effd640 1 -- 172.21.15.98:0/2379923886 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a58031c20 con 0x7f6a680d0a00 2024-09-10T21:00:25.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.664+0000 7f6a6cc23640 1 -- 172.21.15.98:0/2379923886 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a681506b0 con 0x7f6a680d0a00 2024-09-10T21:00:25.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.664+0000 7f6a5effd640 1 -- 172.21.15.98:0/2379923886 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a580330d0 con 0x7f6a680d0a00 2024-09-10T21:00:25.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.664+0000 7f6a6cc23640 1 -- 172.21.15.98:0/2379923886 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a680b3a20 con 0x7f6a680d0a00 2024-09-10T21:00:25.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.664+0000 7f6a6cc23640 1 -- 172.21.15.98:0/2379923886 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6a680cf8e0 con 0x7f6a680d0a00 2024-09-10T21:00:25.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.665+0000 7f6a5effd640 1 -- 172.21.15.98:0/2379923886 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6a58033270 con 0x7f6a680d0a00 2024-09-10T21:00:25.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.666+0000 7f6a5effd640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6a48077f70 0x7f6a4807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:25.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.666+0000 7f6a5effd640 1 -- 172.21.15.98:0/2379923886 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f6a580bb4f0 con 0x7f6a680d0a00 2024-09-10T21:00:25.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.666+0000 7f6a5effd640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.98:6824/3415066506,v1:172.21.15.98:6825/3415066506] conn(0x7f6a4807e4b0 0x7f6a480808f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:25.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.666+0000 7f6a5effd640 1 -- 172.21.15.98:0/2379923886 --> [v2:172.21.15.98:6824/3415066506,v1:172.21.15.98:6825/3415066506] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a580317e0 con 0x7f6a4807e4b0 2024-09-10T21:00:25.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.666+0000 7f6a5effd640 1 -- 172.21.15.98:0/2379923886 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6a580bb980 con 0x7f6a680d0a00 2024-09-10T21:00:25.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.666+0000 7f6a66d9d640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.98:6824/3415066506,v1:172.21.15.98:6825/3415066506] conn(0x7f6a4807e4b0 0x7f6a480808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:25.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.666+0000 7f6a65d9b640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6a48077f70 0x7f6a4807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:25.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.666+0000 7f6a66d9d640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.98:6824/3415066506,v1:172.21.15.98:6825/3415066506] conn(0x7f6a4807e4b0 0x7f6a480808f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:25.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.667+0000 7f6a65d9b640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6a48077f70 0x7f6a4807a430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6a680b4900 tx=0x7f6a50009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:25.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.667+0000 7f6a5effd640 1 -- 172.21.15.98:0/2379923886 <== osd.0 v2:172.21.15.98:6824/3415066506 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6a580317e0 con 0x7f6a4807e4b0 2024-09-10T21:00:25.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.679+0000 7f6a6cc23640 1 -- 172.21.15.98:0/2379923886 --> [v2:172.21.15.98:6824/3415066506,v1:172.21.15.98:6825/3415066506] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6a681553d0 con 0x7f6a4807e4b0 2024-09-10T21:00:25.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.679+0000 7f6a5effd640 1 -- 172.21.15.98:0/2379923886 <== osd.0 v2:172.21.15.98:6824/3415066506 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6a681553d0 con 0x7f6a4807e4b0 2024-09-10T21:00:25.683 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-10T21:00:25.684 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:00:25.684 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-10T21:00:25.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.680+0000 7f6a6cc23640 1 -- 172.21.15.98:0/2379923886 >> [v2:172.21.15.98:6824/3415066506,v1:172.21.15.98:6825/3415066506] conn(0x7f6a4807e4b0 msgr2=0x7f6a480808f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:25.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.680+0000 7f6a6cc23640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.98:6824/3415066506,v1:172.21.15.98:6825/3415066506] conn(0x7f6a4807e4b0 0x7f6a480808f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.680+0000 7f6a6cc23640 1 -- 172.21.15.98:0/2379923886 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6a48077f70 msgr2=0x7f6a4807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:25.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.680+0000 7f6a6cc23640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f6a48077f70 0x7f6a4807a430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6a680b4900 tx=0x7f6a50009040 comp rx=0 tx=0).stop 2024-09-10T21:00:25.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.680+0000 7f6a6cc23640 1 -- 172.21.15.98:0/2379923886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a680d0a00 msgr2=0x7f6a68150170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:25.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.680+0000 7f6a6cc23640 1 --2- 172.21.15.98:0/2379923886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a680d0a00 0x7f6a68150170 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009cd0 tx=0x7f6a580316f0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.680+0000 7f6a66d9d640 1 -- 172.21.15.98:0/2379923886 reap_dead start 2024-09-10T21:00:25.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.680+0000 7f6a6cc23640 1 -- 172.21.15.98:0/2379923886 shutdown_connections 2024-09-10T21:00:25.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.680+0000 7f6a6cc23640 1 -- 172.21.15.98:0/2379923886 >> 172.21.15.98:0/2379923886 conn(0x7f6a680d1d10 msgr2=0x7f6a680ce180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:25.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.680+0000 7f6a6cc23640 1 -- 172.21.15.98:0/2379923886 shutdown_connections 2024-09-10T21:00:25.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.680+0000 7f6a6cc23640 1 -- 172.21.15.98:0/2379923886 wait complete. 2024-09-10T21:00:25.694 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T21:00:25.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.791+0000 7f91ac55d640 1 -- 172.21.15.98:0/1371323325 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a4072ad0 msgr2=0x7f91a40cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:25.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.791+0000 7f91ac55d640 1 --2- 172.21.15.98:0/1371323325 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a4072ad0 0x7f91a40cf040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f919c005fd0 tx=0x7f919c02a820 comp rx=0 tx=0).stop 2024-09-10T21:00:25.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.791+0000 7f91ac55d640 1 -- 172.21.15.98:0/1371323325 shutdown_connections 2024-09-10T21:00:25.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.791+0000 7f91ac55d640 1 --2- 172.21.15.98:0/1371323325 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a4072ad0 0x7f91a40cf040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.791+0000 7f91ac55d640 1 --2- 172.21.15.98:0/1371323325 >> v2:172.21.15.190:3300/0 conn(0x7f91a4072680 0x7f91a40ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.791+0000 7f91ac55d640 1 -- 172.21.15.98:0/1371323325 >> 172.21.15.98:0/1371323325 conn(0x7f91a40b1760 msgr2=0x7f91a40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:25.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.791+0000 7f91ac55d640 1 -- 172.21.15.98:0/1371323325 shutdown_connections 2024-09-10T21:00:25.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.792+0000 7f91ac55d640 1 -- 172.21.15.98:0/1371323325 wait complete. 2024-09-10T21:00:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.793+0000 7f91ac55d640 1 Processor -- start 2024-09-10T21:00:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.793+0000 7f91ac55d640 1 -- start start 2024-09-10T21:00:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.793+0000 7f91ac55d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a4072680 0x7f91a413a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.793+0000 7f91ac55d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91a4072ad0 0x7f91a413e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.793+0000 7f91ac55d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a4144780 0x7f91a415ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.793+0000 7f91ac55d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f91a415c960 con 0x7f91a4144780 2024-09-10T21:00:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.793+0000 7f91ac55d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f91a415c7e0 con 0x7f91a4072ad0 2024-09-10T21:00:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.793+0000 7f91ac55d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f91a415cae0 con 0x7f91a4072680 2024-09-10T21:00:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.793+0000 7f91aaafa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a4144780 0x7f91a415ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f91aaafa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a4144780 0x7f91a415ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41986/0 (socket says 172.21.15.98:41986) 2024-09-10T21:00:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f91aaafa640 1 -- 172.21.15.98:0/2714711488 learned_addr learned my addr 172.21.15.98:0/2714711488 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:25.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f91a9af8640 1 --2- 172.21.15.98:0/2714711488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91a4072ad0 0x7f91a413e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f91aa2f9640 1 --2- 172.21.15.98:0/2714711488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a4072680 0x7f91a413a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f91aaafa640 1 -- 172.21.15.98:0/2714711488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a4072680 msgr2=0x7f91a413a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f91aaafa640 1 --2- 172.21.15.98:0/2714711488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a4072680 0x7f91a413a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f91aaafa640 1 -- 172.21.15.98:0/2714711488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91a4072ad0 msgr2=0x7f91a413e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f91aaafa640 1 --2- 172.21.15.98:0/2714711488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91a4072ad0 0x7f91a413e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f91aaafa640 1 -- 172.21.15.98:0/2714711488 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a41f29a0 con 0x7f91a4144780 2024-09-10T21:00:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f91aaafa640 1 --2- 172.21.15.98:0/2714711488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a4144780 0x7f91a415ea50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f919c035040 tx=0x7f919c032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f919affd640 1 -- 172.21.15.98:0/2714711488 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c037040 con 0x7f91a4144780 2024-09-10T21:00:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f919affd640 1 -- 172.21.15.98:0/2714711488 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f919c042070 con 0x7f91a4144780 2024-09-10T21:00:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f91ac55d640 1 -- 172.21.15.98:0/2714711488 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a41f6400 con 0x7f91a4144780 2024-09-10T21:00:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f919affd640 1 -- 172.21.15.98:0/2714711488 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c003e60 con 0x7f91a4144780 2024-09-10T21:00:25.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.794+0000 7f91ac55d640 1 -- 172.21.15.98:0/2714711488 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a41f69c0 con 0x7f91a4144780 2024-09-10T21:00:25.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.795+0000 7f91ac55d640 1 -- 172.21.15.98:0/2714711488 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a415cfd0 con 0x7f91a4144780 2024-09-10T21:00:25.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.796+0000 7f919affd640 1 -- 172.21.15.98:0/2714711488 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f919c03a030 con 0x7f91a4144780 2024-09-10T21:00:25.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.796+0000 7f919affd640 1 --2- 172.21.15.98:0/2714711488 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9188077fa0 0x7f918807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:25.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.796+0000 7f919affd640 1 -- 172.21.15.98:0/2714711488 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f919c0ba4c0 con 0x7f91a4144780 2024-09-10T21:00:25.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.797+0000 7f91aa2f9640 1 --2- 172.21.15.98:0/2714711488 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9188077fa0 0x7f918807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:25.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.797+0000 7f91aa2f9640 1 --2- 172.21.15.98:0/2714711488 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9188077fa0 0x7f918807a460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f91a0007fd0 tx=0x7f91a00013b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:25.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.801+0000 7f919affd640 1 -- 172.21.15.98:0/2714711488 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f91a41f29a0 con 0x7f91a4144780 2024-09-10T21:00:25.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.973+0000 7f91ac55d640 1 -- 172.21.15.98:0/2714711488 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f91a413e040 con 0x7f9188077fa0 2024-09-10T21:00:25.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.974+0000 7f919affd640 1 -- 172.21.15.98:0/2714711488 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f91a413e040 con 0x7f9188077fa0 2024-09-10T21:00:25.978 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T21:00:25.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.976+0000 7f91ac55d640 1 -- 172.21.15.98:0/2714711488 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9188077fa0 msgr2=0x7f918807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:25.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.976+0000 7f91ac55d640 1 --2- 172.21.15.98:0/2714711488 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9188077fa0 0x7f918807a460 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f91a0007fd0 tx=0x7f91a00013b0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.976+0000 7f91ac55d640 1 -- 172.21.15.98:0/2714711488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a4144780 msgr2=0x7f91a415ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:25.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.976+0000 7f91ac55d640 1 --2- 172.21.15.98:0/2714711488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a4144780 0x7f91a415ea50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f919c035040 tx=0x7f919c032c60 comp rx=0 tx=0).stop 2024-09-10T21:00:25.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.976+0000 7f91ac55d640 1 -- 172.21.15.98:0/2714711488 shutdown_connections 2024-09-10T21:00:25.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.976+0000 7f91ac55d640 1 --2- 172.21.15.98:0/2714711488 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f9188077fa0 0x7f918807a460 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.976+0000 7f91ac55d640 1 --2- 172.21.15.98:0/2714711488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a4144780 0x7f91a415ea50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.976+0000 7f91ac55d640 1 --2- 172.21.15.98:0/2714711488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91a4072ad0 0x7f91a413e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.976+0000 7f91ac55d640 1 --2- 172.21.15.98:0/2714711488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f91a4072680 0x7f91a413a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:25.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.976+0000 7f91ac55d640 1 -- 172.21.15.98:0/2714711488 >> 172.21.15.98:0/2714711488 conn(0x7f91a40b1760 msgr2=0x7f91a40ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:25.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.976+0000 7f91ac55d640 1 -- 172.21.15.98:0/2714711488 shutdown_connections 2024-09-10T21:00:25.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:25.976+0000 7f91ac55d640 1 -- 172.21.15.98:0/2714711488 wait complete. 2024-09-10T21:00:25.990 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T21:00:25.991 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:00:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.092+0000 7f294357d640 1 -- 172.21.15.98:0/1574982530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f293c15bdb0 msgr2=0x7f293c0b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.092+0000 7f294357d640 1 --2- 172.21.15.98:0/1574982530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f293c15bdb0 0x7f293c0b1830 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2938009830 tx=0x7f293802a510 comp rx=0 tx=0).stop 2024-09-10T21:00:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.092+0000 7f294357d640 1 -- 172.21.15.98:0/1574982530 shutdown_connections 2024-09-10T21:00:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.092+0000 7f294357d640 1 --2- 172.21.15.98:0/1574982530 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f293c074c70 0x7f293c1600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.092+0000 7f294357d640 1 --2- 172.21.15.98:0/1574982530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f293c15bdb0 0x7f293c0b1830 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.092+0000 7f294357d640 1 -- 172.21.15.98:0/1574982530 >> 172.21.15.98:0/1574982530 conn(0x7f293c074150 msgr2=0x7f293c074560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.092+0000 7f294357d640 1 -- 172.21.15.98:0/1574982530 shutdown_connections 2024-09-10T21:00:26.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.092+0000 7f294357d640 1 -- 172.21.15.98:0/1574982530 wait complete. 2024-09-10T21:00:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.093+0000 7f294357d640 1 Processor -- start 2024-09-10T21:00:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.093+0000 7f294357d640 1 -- start start 2024-09-10T21:00:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.094+0000 7f294357d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f293c06ca50 0x7f293c1f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.094+0000 7f294357d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f293c074c70 0x7f293c0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.094+0000 7f294357d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f293c15bdb0 0x7f293c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.094+0000 7f294357d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f293c06d290 con 0x7f293c06ca50 2024-09-10T21:00:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.094+0000 7f294357d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f293c06d110 con 0x7f293c15bdb0 2024-09-10T21:00:26.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.094+0000 7f294357d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f293c160660 con 0x7f293c074c70 2024-09-10T21:00:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.094+0000 7f2941319640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f293c06ca50 0x7f293c1f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.094+0000 7f2941319640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f293c06ca50 0x7f293c1f6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41996/0 (socket says 172.21.15.98:41996) 2024-09-10T21:00:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.094+0000 7f2941319640 1 -- 172.21.15.98:0/2645164457 learned_addr learned my addr 172.21.15.98:0/2645164457 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.094+0000 7f2940b18640 1 --2- 172.21.15.98:0/2645164457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f293c074c70 0x7f293c0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.094+0000 7f2941b1a640 1 --2- 172.21.15.98:0/2645164457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f293c15bdb0 0x7f293c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.095+0000 7f2941319640 1 -- 172.21.15.98:0/2645164457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f293c074c70 msgr2=0x7f293c0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.095+0000 7f2941319640 1 --2- 172.21.15.98:0/2645164457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f293c074c70 0x7f293c0c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.095+0000 7f2941319640 1 -- 172.21.15.98:0/2645164457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f293c15bdb0 msgr2=0x7f293c159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.095+0000 7f2941319640 1 --2- 172.21.15.98:0/2645164457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f293c15bdb0 0x7f293c159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.095+0000 7f2941319640 1 -- 172.21.15.98:0/2645164457 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293c1e79c0 con 0x7f293c06ca50 2024-09-10T21:00:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.095+0000 7f2941319640 1 --2- 172.21.15.98:0/2645164457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f293c06ca50 0x7f293c1f6b60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f293802ab00 tx=0x7f2938033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:26.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.095+0000 7f2931ffb640 1 -- 172.21.15.98:0/2645164457 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938031d60 con 0x7f293c06ca50 2024-09-10T21:00:26.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.095+0000 7f2931ffb640 1 -- 172.21.15.98:0/2645164457 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2938033e10 con 0x7f293c06ca50 2024-09-10T21:00:26.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.095+0000 7f2931ffb640 1 -- 172.21.15.98:0/2645164457 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938005260 con 0x7f293c06ca50 2024-09-10T21:00:26.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.095+0000 7f294357d640 1 -- 172.21.15.98:0/2645164457 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293c1e7ca0 con 0x7f293c06ca50 2024-09-10T21:00:26.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.095+0000 7f294357d640 1 -- 172.21.15.98:0/2645164457 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293c0c70d0 con 0x7f293c06ca50 2024-09-10T21:00:26.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.096+0000 7f294357d640 1 -- 172.21.15.98:0/2645164457 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f293c1e7ee0 con 0x7f293c06ca50 2024-09-10T21:00:26.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.097+0000 7f2931ffb640 1 -- 172.21.15.98:0/2645164457 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2938005400 con 0x7f293c06ca50 2024-09-10T21:00:26.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.097+0000 7f2931ffb640 1 --2- 172.21.15.98:0/2645164457 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2920077f70 0x7f292007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.097+0000 7f2931ffb640 1 -- 172.21.15.98:0/2645164457 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f29380bab70 con 0x7f293c06ca50 2024-09-10T21:00:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.097+0000 7f2931ffb640 1 --2- 172.21.15.98:0/2645164457 >> [v2:172.21.15.98:6800/3091760272,v1:172.21.15.98:6801/3091760272] conn(0x7f292007e4b0 0x7f29200808f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.097+0000 7f2931ffb640 1 -- 172.21.15.98:0/2645164457 --> [v2:172.21.15.98:6800/3091760272,v1:172.21.15.98:6801/3091760272] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f29380317a0 con 0x7f292007e4b0 2024-09-10T21:00:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.097+0000 7f2931ffb640 1 -- 172.21.15.98:0/2645164457 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f29380bf2c0 con 0x7f293c06ca50 2024-09-10T21:00:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.097+0000 7f2940b18640 1 --2- 172.21.15.98:0/2645164457 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2920077f70 0x7f292007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.098+0000 7f2941b1a640 1 --2- 172.21.15.98:0/2645164457 >> [v2:172.21.15.98:6800/3091760272,v1:172.21.15.98:6801/3091760272] conn(0x7f292007e4b0 0x7f29200808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.098+0000 7f2941b1a640 1 --2- 172.21.15.98:0/2645164457 >> [v2:172.21.15.98:6800/3091760272,v1:172.21.15.98:6801/3091760272] conn(0x7f292007e4b0 0x7f29200808f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.098+0000 7f2940b18640 1 --2- 172.21.15.98:0/2645164457 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2920077f70 0x7f292007a430 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f293c1f6120 tx=0x7f292c009260 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:26.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.098+0000 7f2931ffb640 1 -- 172.21.15.98:0/2645164457 <== osd.1 v2:172.21.15.98:6800/3091760272 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f29380317a0 con 0x7f292007e4b0 2024-09-10T21:00:26.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.111+0000 7f294357d640 1 -- 172.21.15.98:0/2645164457 --> [v2:172.21.15.98:6800/3091760272,v1:172.21.15.98:6801/3091760272] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f293c0b59f0 con 0x7f292007e4b0 2024-09-10T21:00:26.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.111+0000 7f2931ffb640 1 -- 172.21.15.98:0/2645164457 <== osd.1 v2:172.21.15.98:6800/3091760272 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f293c0b59f0 con 0x7f292007e4b0 2024-09-10T21:00:26.115 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-10T21:00:26.115 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:00:26.116 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-10T21:00:26.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.111+0000 7f294357d640 1 -- 172.21.15.98:0/2645164457 >> [v2:172.21.15.98:6800/3091760272,v1:172.21.15.98:6801/3091760272] conn(0x7f292007e4b0 msgr2=0x7f29200808f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.111+0000 7f294357d640 1 --2- 172.21.15.98:0/2645164457 >> [v2:172.21.15.98:6800/3091760272,v1:172.21.15.98:6801/3091760272] conn(0x7f292007e4b0 0x7f29200808f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.111+0000 7f294357d640 1 -- 172.21.15.98:0/2645164457 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2920077f70 msgr2=0x7f292007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.111+0000 7f294357d640 1 --2- 172.21.15.98:0/2645164457 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2920077f70 0x7f292007a430 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f293c1f6120 tx=0x7f292c009260 comp rx=0 tx=0).stop 2024-09-10T21:00:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.111+0000 7f294357d640 1 -- 172.21.15.98:0/2645164457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f293c06ca50 msgr2=0x7f293c1f6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.111+0000 7f294357d640 1 --2- 172.21.15.98:0/2645164457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f293c06ca50 0x7f293c1f6b60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f293802ab00 tx=0x7f2938033c20 comp rx=0 tx=0).stop 2024-09-10T21:00:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.111+0000 7f2941b1a640 1 -- 172.21.15.98:0/2645164457 reap_dead start 2024-09-10T21:00:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.111+0000 7f294357d640 1 -- 172.21.15.98:0/2645164457 shutdown_connections 2024-09-10T21:00:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.112+0000 7f294357d640 1 -- 172.21.15.98:0/2645164457 >> 172.21.15.98:0/2645164457 conn(0x7f293c074150 msgr2=0x7f293c06fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.112+0000 7f294357d640 1 -- 172.21.15.98:0/2645164457 shutdown_connections 2024-09-10T21:00:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.112+0000 7f294357d640 1 -- 172.21.15.98:0/2645164457 wait complete. 2024-09-10T21:00:26.125 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T21:00:26.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.223+0000 7f2f1d07f640 1 -- 172.21.15.98:0/2323204912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f180cf210 msgr2=0x7f2f1815c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.223+0000 7f2f1d07f640 1 --2- 172.21.15.98:0/2323204912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f180cf210 0x7f2f1815c550 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009830 tx=0x7f2f1002a480 comp rx=0 tx=0).stop 2024-09-10T21:00:26.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.223+0000 7f2f1d07f640 1 -- 172.21.15.98:0/2323204912 shutdown_connections 2024-09-10T21:00:26.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.223+0000 7f2f1d07f640 1 --2- 172.21.15.98:0/2323204912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f180cf210 0x7f2f1815c550 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.223+0000 7f2f1d07f640 1 -- 172.21.15.98:0/2323204912 >> 172.21.15.98:0/2323204912 conn(0x7f2f180b1760 msgr2=0x7f2f180b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:26.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.223+0000 7f2f1d07f640 1 -- 172.21.15.98:0/2323204912 shutdown_connections 2024-09-10T21:00:26.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.223+0000 7f2f1d07f640 1 -- 172.21.15.98:0/2323204912 wait complete. 2024-09-10T21:00:26.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.224+0000 7f2f1d07f640 1 Processor -- start 2024-09-10T21:00:26.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.224+0000 7f2f1d07f640 1 -- start start 2024-09-10T21:00:26.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1d07f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f18072680 0x7f2f181f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1d07f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f180cf210 0x7f2f1815a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1d07f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f18144780 0x7f2f181f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1d07f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1815cc10 con 0x7f2f180cf210 2024-09-10T21:00:26.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1d07f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1815ca90 con 0x7f2f18072680 2024-09-10T21:00:26.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1d07f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2f1815cd90 con 0x7f2f18144780 2024-09-10T21:00:26.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f180cf210 0x7f2f1815a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f180cf210 0x7f2f1815a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42006/0 (socket says 172.21.15.98:42006) 2024-09-10T21:00:26.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1659c640 1 -- 172.21.15.98:0/1810516048 learned_addr learned my addr 172.21.15.98:0/1810516048 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:26.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f16d9d640 1 --2- 172.21.15.98:0/1810516048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f18072680 0x7f2f181f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1759e640 1 --2- 172.21.15.98:0/1810516048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f18144780 0x7f2f181f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1659c640 1 -- 172.21.15.98:0/1810516048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f18144780 msgr2=0x7f2f181f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1659c640 1 --2- 172.21.15.98:0/1810516048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f18144780 0x7f2f181f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1659c640 1 -- 172.21.15.98:0/1810516048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f18072680 msgr2=0x7f2f181f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1659c640 1 --2- 172.21.15.98:0/1810516048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f18072680 0x7f2f181f2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.225+0000 7f2f1659c640 1 -- 172.21.15.98:0/1810516048 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f181f5c50 con 0x7f2f180cf210 2024-09-10T21:00:26.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.226+0000 7f2f1659c640 1 --2- 172.21.15.98:0/1810516048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f180cf210 0x7f2f1815a530 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00c500 tx=0x7f2f0c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:26.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.226+0000 7f2eff7fe640 1 -- 172.21.15.98:0/1810516048 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c016020 con 0x7f2f180cf210 2024-09-10T21:00:26.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.226+0000 7f2eff7fe640 1 -- 172.21.15.98:0/1810516048 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f0c012920 con 0x7f2f180cf210 2024-09-10T21:00:26.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.226+0000 7f2f1d07f640 1 -- 172.21.15.98:0/1810516048 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f181f5f30 con 0x7f2f180cf210 2024-09-10T21:00:26.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.226+0000 7f2eff7fe640 1 -- 172.21.15.98:0/1810516048 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c012d90 con 0x7f2f180cf210 2024-09-10T21:00:26.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.226+0000 7f2f1d07f640 1 -- 172.21.15.98:0/1810516048 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f181f1e90 con 0x7f2f180cf210 2024-09-10T21:00:26.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.227+0000 7f2f1d07f640 1 -- 172.21.15.98:0/1810516048 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f180b1f90 con 0x7f2f180cf210 2024-09-10T21:00:26.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.227+0000 7f2eff7fe640 1 -- 172.21.15.98:0/1810516048 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2f0c01a3b0 con 0x7f2f180cf210 2024-09-10T21:00:26.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.228+0000 7f2eff7fe640 1 --2- 172.21.15.98:0/1810516048 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2f04077f70 0x7f2f0407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.228+0000 7f2eff7fe640 1 -- 172.21.15.98:0/1810516048 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f2f0c099cb0 con 0x7f2f180cf210 2024-09-10T21:00:26.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.228+0000 7f2f16d9d640 1 --2- 172.21.15.98:0/1810516048 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2f04077f70 0x7f2f0407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.229+0000 7f2f16d9d640 1 --2- 172.21.15.98:0/1810516048 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2f04077f70 0x7f2f0407a430 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009f20 tx=0x7f2f080094d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:26.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.232+0000 7f2eff7fe640 1 -- 172.21.15.98:0/1810516048 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2f180b1f90 con 0x7f2f180cf210 2024-09-10T21:00:26.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.408+0000 7f2f1d07f640 1 -- 172.21.15.98:0/1810516048 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f2f181442c0 con 0x7f2f04077f70 2024-09-10T21:00:26.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.409+0000 7f2eff7fe640 1 -- 172.21.15.98:0/1810516048 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2f181442c0 con 0x7f2f04077f70 2024-09-10T21:00:26.413 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T21:00:26.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.411+0000 7f2f1d07f640 1 -- 172.21.15.98:0/1810516048 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2f04077f70 msgr2=0x7f2f0407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.411+0000 7f2f1d07f640 1 --2- 172.21.15.98:0/1810516048 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2f04077f70 0x7f2f0407a430 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009f20 tx=0x7f2f080094d0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.411+0000 7f2f1d07f640 1 -- 172.21.15.98:0/1810516048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f180cf210 msgr2=0x7f2f1815a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.411+0000 7f2f1d07f640 1 --2- 172.21.15.98:0/1810516048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f180cf210 0x7f2f1815a530 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00c500 tx=0x7f2f0c00c9c0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.411+0000 7f2f1d07f640 1 -- 172.21.15.98:0/1810516048 shutdown_connections 2024-09-10T21:00:26.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.411+0000 7f2f1d07f640 1 --2- 172.21.15.98:0/1810516048 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2f04077f70 0x7f2f0407a430 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.411+0000 7f2f1d07f640 1 --2- 172.21.15.98:0/1810516048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f18144780 0x7f2f181f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.411+0000 7f2f1d07f640 1 --2- 172.21.15.98:0/1810516048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f180cf210 0x7f2f1815a530 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.411+0000 7f2f1d07f640 1 --2- 172.21.15.98:0/1810516048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f18072680 0x7f2f181f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.411+0000 7f2f1d07f640 1 -- 172.21.15.98:0/1810516048 >> 172.21.15.98:0/1810516048 conn(0x7f2f180b1760 msgr2=0x7f2f180b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:26.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.411+0000 7f2f1d07f640 1 -- 172.21.15.98:0/1810516048 shutdown_connections 2024-09-10T21:00:26.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.411+0000 7f2f1d07f640 1 -- 172.21.15.98:0/1810516048 wait complete. 2024-09-10T21:00:26.426 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T21:00:26.426 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:00:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.527+0000 7f8e40afe640 1 -- 172.21.15.98:0/428236929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e28003a30 msgr2=0x7f8e28003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.527+0000 7f8e40afe640 1 --2- 172.21.15.98:0/428236929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e28003a30 0x7f8e28003e50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8e340016c0 tx=0x7f8e28005830 comp rx=0 tx=0).stop 2024-09-10T21:00:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.527+0000 7f8e3ad9d640 1 -- 172.21.15.98:0/428236929 reap_dead start 2024-09-10T21:00:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.527+0000 7f8e40afe640 1 -- 172.21.15.98:0/428236929 shutdown_connections 2024-09-10T21:00:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.527+0000 7f8e40afe640 1 -- 172.21.15.98:0/428236929 >> 172.21.15.98:0/428236929 conn(0x7f8e3c0d3f70 msgr2=0x7f8e3c0d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.527+0000 7f8e40afe640 1 -- 172.21.15.98:0/428236929 shutdown_connections 2024-09-10T21:00:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.527+0000 7f8e40afe640 1 -- 172.21.15.98:0/428236929 wait complete. 2024-09-10T21:00:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.528+0000 7f8e40afe640 1 Processor -- start 2024-09-10T21:00:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.528+0000 7f8e40afe640 1 -- start start 2024-09-10T21:00:26.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e40afe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e28003a30 0x7f8e3c153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e40afe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e3c15f280 0x7f8e3c156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e40afe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e3c160f90 0x7f8e3c156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e40afe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c0d2660 con 0x7f8e3c15f280 2024-09-10T21:00:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e40afe640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c0d24e0 con 0x7f8e3c160f90 2024-09-10T21:00:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e40afe640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3c0d27e0 con 0x7f8e28003a30 2024-09-10T21:00:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e39d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e3c15f280 0x7f8e3c156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e39d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e3c15f280 0x7f8e3c156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42028/0 (socket says 172.21.15.98:42028) 2024-09-10T21:00:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e39d9b640 1 -- 172.21.15.98:0/3059300553 learned_addr learned my addr 172.21.15.98:0/3059300553 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e3a59c640 1 --2- 172.21.15.98:0/3059300553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e28003a30 0x7f8e3c153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e3ad9d640 1 --2- 172.21.15.98:0/3059300553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e3c160f90 0x7f8e3c156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e39d9b640 1 -- 172.21.15.98:0/3059300553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e28003a30 msgr2=0x7f8e3c153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e39d9b640 1 --2- 172.21.15.98:0/3059300553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e28003a30 0x7f8e3c153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e39d9b640 1 -- 172.21.15.98:0/3059300553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e3c160f90 msgr2=0x7f8e3c156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e39d9b640 1 --2- 172.21.15.98:0/3059300553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e3c160f90 0x7f8e3c156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.529+0000 7f8e39d9b640 1 -- 172.21.15.98:0/3059300553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3c1574c0 con 0x7f8e3c15f280 2024-09-10T21:00:26.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.530+0000 7f8e39d9b640 1 --2- 172.21.15.98:0/3059300553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e3c15f280 0x7f8e3c156770 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400fa00 tx=0x7f8e2400fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:26.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.530+0000 7f8e32ffd640 1 -- 172.21.15.98:0/3059300553 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2400bcb0 con 0x7f8e3c15f280 2024-09-10T21:00:26.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.530+0000 7f8e32ffd640 1 -- 172.21.15.98:0/3059300553 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e24004030 con 0x7f8e3c15f280 2024-09-10T21:00:26.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.530+0000 7f8e40afe640 1 -- 172.21.15.98:0/3059300553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3c15fbb0 con 0x7f8e3c15f280 2024-09-10T21:00:26.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.530+0000 7f8e32ffd640 1 -- 172.21.15.98:0/3059300553 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24004480 con 0x7f8e3c15f280 2024-09-10T21:00:26.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.530+0000 7f8e40afe640 1 -- 172.21.15.98:0/3059300553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e3c160170 con 0x7f8e3c15f280 2024-09-10T21:00:26.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.530+0000 7f8e40afe640 1 -- 172.21.15.98:0/3059300553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8e3c15f960 con 0x7f8e3c15f280 2024-09-10T21:00:26.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.531+0000 7f8e32ffd640 1 -- 172.21.15.98:0/3059300553 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8e240035f0 con 0x7f8e3c15f280 2024-09-10T21:00:26.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.532+0000 7f8e32ffd640 1 --2- 172.21.15.98:0/3059300553 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8e1c077f70 0x7f8e1c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.532+0000 7f8e32ffd640 1 -- 172.21.15.98:0/3059300553 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f8e2409a400 con 0x7f8e3c15f280 2024-09-10T21:00:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.532+0000 7f8e32ffd640 1 --2- 172.21.15.98:0/3059300553 >> [v2:172.21.15.98:6808/2541446963,v1:172.21.15.98:6809/2541446963] conn(0x7f8e1c07e4b0 0x7f8e1c0808f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.532+0000 7f8e32ffd640 1 -- 172.21.15.98:0/3059300553 --> [v2:172.21.15.98:6808/2541446963,v1:172.21.15.98:6809/2541446963] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e24019ec0 con 0x7f8e1c07e4b0 2024-09-10T21:00:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.532+0000 7f8e32ffd640 1 -- 172.21.15.98:0/3059300553 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8e240a1050 con 0x7f8e3c15f280 2024-09-10T21:00:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.532+0000 7f8e3a59c640 1 --2- 172.21.15.98:0/3059300553 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8e1c077f70 0x7f8e1c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.532+0000 7f8e3ad9d640 1 --2- 172.21.15.98:0/3059300553 >> [v2:172.21.15.98:6808/2541446963,v1:172.21.15.98:6809/2541446963] conn(0x7f8e1c07e4b0 0x7f8e1c0808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.532+0000 7f8e3ad9d640 1 --2- 172.21.15.98:0/3059300553 >> [v2:172.21.15.98:6808/2541446963,v1:172.21.15.98:6809/2541446963] conn(0x7f8e1c07e4b0 0x7f8e1c0808f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.532+0000 7f8e3a59c640 1 --2- 172.21.15.98:0/3059300553 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8e1c077f70 0x7f8e1c07a430 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8e34032000 tx=0x7f8e3402c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.533+0000 7f8e32ffd640 1 -- 172.21.15.98:0/3059300553 <== osd.2 v2:172.21.15.98:6808/2541446963 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e24019ec0 con 0x7f8e1c07e4b0 2024-09-10T21:00:26.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.545+0000 7f8e40afe640 1 -- 172.21.15.98:0/3059300553 --> [v2:172.21.15.98:6808/2541446963,v1:172.21.15.98:6809/2541446963] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8e3c0cea40 con 0x7f8e1c07e4b0 2024-09-10T21:00:26.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.545+0000 7f8e32ffd640 1 -- 172.21.15.98:0/3059300553 <== osd.2 v2:172.21.15.98:6808/2541446963 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8e3c0cea40 con 0x7f8e1c07e4b0 2024-09-10T21:00:26.550 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-10T21:00:26.550 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:00:26.550 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-10T21:00:26.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.546+0000 7f8e40afe640 1 -- 172.21.15.98:0/3059300553 >> [v2:172.21.15.98:6808/2541446963,v1:172.21.15.98:6809/2541446963] conn(0x7f8e1c07e4b0 msgr2=0x7f8e1c0808f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.546+0000 7f8e40afe640 1 --2- 172.21.15.98:0/3059300553 >> [v2:172.21.15.98:6808/2541446963,v1:172.21.15.98:6809/2541446963] conn(0x7f8e1c07e4b0 0x7f8e1c0808f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.546+0000 7f8e40afe640 1 -- 172.21.15.98:0/3059300553 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8e1c077f70 msgr2=0x7f8e1c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.546+0000 7f8e40afe640 1 --2- 172.21.15.98:0/3059300553 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8e1c077f70 0x7f8e1c07a430 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8e34032000 tx=0x7f8e3402c040 comp rx=0 tx=0).stop 2024-09-10T21:00:26.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.546+0000 7f8e40afe640 1 -- 172.21.15.98:0/3059300553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e3c15f280 msgr2=0x7f8e3c156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.546+0000 7f8e40afe640 1 --2- 172.21.15.98:0/3059300553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e3c15f280 0x7f8e3c156770 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400fa00 tx=0x7f8e2400fe00 comp rx=0 tx=0).stop 2024-09-10T21:00:26.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.546+0000 7f8e3ad9d640 1 -- 172.21.15.98:0/3059300553 reap_dead start 2024-09-10T21:00:26.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.546+0000 7f8e40afe640 1 -- 172.21.15.98:0/3059300553 shutdown_connections 2024-09-10T21:00:26.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.546+0000 7f8e40afe640 1 -- 172.21.15.98:0/3059300553 >> 172.21.15.98:0/3059300553 conn(0x7f8e3c0d3f70 msgr2=0x7f8e3c0cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:26.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.546+0000 7f8e40afe640 1 -- 172.21.15.98:0/3059300553 shutdown_connections 2024-09-10T21:00:26.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.546+0000 7f8e40afe640 1 -- 172.21.15.98:0/3059300553 wait complete. 2024-09-10T21:00:26.560 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T21:00:26.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.658+0000 7f8a1aabf640 1 -- 172.21.15.98:0/2041229299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a1406f140 msgr2=0x7f8a1406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.658+0000 7f8a1aabf640 1 --2- 172.21.15.98:0/2041229299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a1406f140 0x7f8a1406f5d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c005fd0 tx=0x7f8a0c02a8b0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.658+0000 7f8a1aabf640 1 -- 172.21.15.98:0/2041229299 shutdown_connections 2024-09-10T21:00:26.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.658+0000 7f8a1aabf640 1 --2- 172.21.15.98:0/2041229299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a1406f140 0x7f8a1406f5d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.658+0000 7f8a1aabf640 1 --2- 172.21.15.98:0/2041229299 >> v2:172.21.15.190:3300/0 conn(0x7f8a14073780 0x7f8a14073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.658+0000 7f8a1aabf640 1 -- 172.21.15.98:0/2041229299 >> 172.21.15.98:0/2041229299 conn(0x7f8a140b1760 msgr2=0x7f8a140b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:26.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.658+0000 7f8a1aabf640 1 -- 172.21.15.98:0/2041229299 shutdown_connections 2024-09-10T21:00:26.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.658+0000 7f8a1aabf640 1 -- 172.21.15.98:0/2041229299 wait complete. 2024-09-10T21:00:26.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.659+0000 7f8a1aabf640 1 Processor -- start 2024-09-10T21:00:26.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.659+0000 7f8a1aabf640 1 -- start start 2024-09-10T21:00:26.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1aabf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a1406f140 0x7f8a141f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1aabf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a14073780 0x7f8a140b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1aabf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a1415c470 0x7f8a141f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1aabf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14142260 con 0x7f8a1406f140 2024-09-10T21:00:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1aabf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141420e0 con 0x7f8a1415c470 2024-09-10T21:00:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1aabf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1406fb10 con 0x7f8a14073780 2024-09-10T21:00:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1885b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a1406f140 0x7f8a141f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1885b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a1406f140 0x7f8a141f53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42030/0 (socket says 172.21.15.98:42030) 2024-09-10T21:00:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1885b640 1 -- 172.21.15.98:0/3353296136 learned_addr learned my addr 172.21.15.98:0/3353296136 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a0bfff640 1 --2- 172.21.15.98:0/3353296136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a14073780 0x7f8a140b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1905c640 1 --2- 172.21.15.98:0/3353296136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a1415c470 0x7f8a141f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1885b640 1 -- 172.21.15.98:0/3353296136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a14073780 msgr2=0x7f8a140b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1885b640 1 --2- 172.21.15.98:0/3353296136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a14073780 0x7f8a140b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1885b640 1 -- 172.21.15.98:0/3353296136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a1415c470 msgr2=0x7f8a141f7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1885b640 1 --2- 172.21.15.98:0/3353296136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a1415c470 0x7f8a141f7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.660+0000 7f8a1885b640 1 -- 172.21.15.98:0/3353296136 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a14144050 con 0x7f8a1406f140 2024-09-10T21:00:26.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.661+0000 7f8a1885b640 1 --2- 172.21.15.98:0/3353296136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a1406f140 0x7f8a141f53d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000e9c0 tx=0x7f8a1000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:26.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.661+0000 7f8a097fa640 1 -- 172.21.15.98:0/3353296136 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000cde0 con 0x7f8a1406f140 2024-09-10T21:00:26.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.661+0000 7f8a1aabf640 1 -- 172.21.15.98:0/3353296136 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a141442c0 con 0x7f8a1406f140 2024-09-10T21:00:26.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.661+0000 7f8a097fa640 1 -- 172.21.15.98:0/3353296136 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a10004020 con 0x7f8a1406f140 2024-09-10T21:00:26.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.661+0000 7f8a097fa640 1 -- 172.21.15.98:0/3353296136 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10004450 con 0x7f8a1406f140 2024-09-10T21:00:26.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.661+0000 7f8a1aabf640 1 -- 172.21.15.98:0/3353296136 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a14144880 con 0x7f8a1406f140 2024-09-10T21:00:26.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.662+0000 7f8a1aabf640 1 -- 172.21.15.98:0/3353296136 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a140700a0 con 0x7f8a1406f140 2024-09-10T21:00:26.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.662+0000 7f8a097fa640 1 -- 172.21.15.98:0/3353296136 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8a100035f0 con 0x7f8a1406f140 2024-09-10T21:00:26.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.663+0000 7f8a097fa640 1 --2- 172.21.15.98:0/3353296136 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8a00077f70 0x7f8a0007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.663+0000 7f8a097fa640 1 -- 172.21.15.98:0/3353296136 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f8a1009a4c0 con 0x7f8a1406f140 2024-09-10T21:00:26.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.663+0000 7f8a0bfff640 1 --2- 172.21.15.98:0/3353296136 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8a00077f70 0x7f8a0007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.664+0000 7f8a0bfff640 1 --2- 172.21.15.98:0/3353296136 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8a00077f70 0x7f8a0007a430 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8a04001720 tx=0x7f8a04009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:26.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.667+0000 7f8a097fa640 1 -- 172.21.15.98:0/3353296136 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8a10062aa0 con 0x7f8a1406f140 2024-09-10T21:00:26.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.843+0000 7f8a1aabf640 1 -- 172.21.15.98:0/3353296136 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f8a14144c80 con 0x7f8a00077f70 2024-09-10T21:00:26.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.844+0000 7f8a097fa640 1 -- 172.21.15.98:0/3353296136 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8a14144c80 con 0x7f8a00077f70 2024-09-10T21:00:26.848 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T21:00:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.846+0000 7f8a1aabf640 1 -- 172.21.15.98:0/3353296136 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8a00077f70 msgr2=0x7f8a0007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.846+0000 7f8a1aabf640 1 --2- 172.21.15.98:0/3353296136 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8a00077f70 0x7f8a0007a430 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8a04001720 tx=0x7f8a04009040 comp rx=0 tx=0).stop 2024-09-10T21:00:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.846+0000 7f8a1aabf640 1 -- 172.21.15.98:0/3353296136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a1406f140 msgr2=0x7f8a141f53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.846+0000 7f8a1aabf640 1 --2- 172.21.15.98:0/3353296136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a1406f140 0x7f8a141f53d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000e9c0 tx=0x7f8a1000ee80 comp rx=0 tx=0).stop 2024-09-10T21:00:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.846+0000 7f8a1aabf640 1 -- 172.21.15.98:0/3353296136 shutdown_connections 2024-09-10T21:00:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.846+0000 7f8a1aabf640 1 --2- 172.21.15.98:0/3353296136 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f8a00077f70 0x7f8a0007a430 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.846+0000 7f8a1aabf640 1 --2- 172.21.15.98:0/3353296136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a1415c470 0x7f8a141f7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.846+0000 7f8a1aabf640 1 --2- 172.21.15.98:0/3353296136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a14073780 0x7f8a140b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.846+0000 7f8a1aabf640 1 --2- 172.21.15.98:0/3353296136 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a1406f140 0x7f8a141f53d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.846+0000 7f8a1aabf640 1 -- 172.21.15.98:0/3353296136 >> 172.21.15.98:0/3353296136 conn(0x7f8a140b1760 msgr2=0x7f8a140cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.846+0000 7f8a1aabf640 1 -- 172.21.15.98:0/3353296136 shutdown_connections 2024-09-10T21:00:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.846+0000 7f8a1aabf640 1 -- 172.21.15.98:0/3353296136 wait complete. 2024-09-10T21:00:26.861 INFO:tasks.ceph:Scrubbing osd.3 2024-09-10T21:00:26.861 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:00:26.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.962+0000 7f1b01448640 1 -- 172.21.15.98:0/869051813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae8004800 msgr2=0x7f1ae8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.962+0000 7f1b01448640 1 --2- 172.21.15.98:0/869051813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae8004800 0x7f1ae8004c80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4006fa0 tx=0x7f1ae8005830 comp rx=0 tx=0).stop 2024-09-10T21:00:26.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.962+0000 7f1b01448640 1 -- 172.21.15.98:0/869051813 shutdown_connections 2024-09-10T21:00:26.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.962+0000 7f1b01448640 1 --2- 172.21.15.98:0/869051813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ae8004800 0x7f1ae8004c80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.962+0000 7f1b01448640 1 -- 172.21.15.98:0/869051813 >> 172.21.15.98:0/869051813 conn(0x7f1afc09d170 msgr2=0x7f1afc0d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:26.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.963+0000 7f1b01448640 1 -- 172.21.15.98:0/869051813 shutdown_connections 2024-09-10T21:00:26.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.963+0000 7f1b01448640 1 -- 172.21.15.98:0/869051813 wait complete. 2024-09-10T21:00:26.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.964+0000 7f1b01448640 1 Processor -- start 2024-09-10T21:00:26.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.964+0000 7f1b01448640 1 -- start start 2024-09-10T21:00:26.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1b01448640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae8004800 0x7f1afc09dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1b01448640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1afc14e070 0x7f1afc14e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1b01448640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1afc0b1610 0x7f1afc0b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1b01448640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc0d2360 con 0x7f1afc14e070 2024-09-10T21:00:26.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1b01448640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc0d4830 con 0x7f1ae8004800 2024-09-10T21:00:26.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1b01448640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1afc0d24e0 con 0x7f1afc0b1610 2024-09-10T21:00:26.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1afa7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1afc14e070 0x7f1afc14e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1afa7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1afc14e070 0x7f1afc14e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42036/0 (socket says 172.21.15.98:42036) 2024-09-10T21:00:26.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1afa7fc640 1 -- 172.21.15.98:0/3833563155 learned_addr learned my addr 172.21.15.98:0/3833563155 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:26.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1afaffd640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae8004800 0x7f1afc09dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1afb7fe640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1afc0b1610 0x7f1afc0b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1afa7fc640 1 -- 172.21.15.98:0/3833563155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1afc0b1610 msgr2=0x7f1afc0b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1afa7fc640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1afc0b1610 0x7f1afc0b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1afa7fc640 1 -- 172.21.15.98:0/3833563155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae8004800 msgr2=0x7f1afc09dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1afa7fc640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae8004800 0x7f1afc09dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1afa7fc640 1 -- 172.21.15.98:0/3833563155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc0b2380 con 0x7f1afc14e070 2024-09-10T21:00:26.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1afaffd640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ae8004800 0x7f1afc09dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:26.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1afa7fc640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1afc14e070 0x7f1afc14e4d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400c320 tx=0x7f1ae4001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:26.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1af37fe640 1 -- 172.21.15.98:0/3833563155 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4003a00 con 0x7f1afc14e070 2024-09-10T21:00:26.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1af37fe640 1 -- 172.21.15.98:0/3833563155 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ae4003c90 con 0x7f1afc14e070 2024-09-10T21:00:26.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1af37fe640 1 -- 172.21.15.98:0/3833563155 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae402e6a0 con 0x7f1afc14e070 2024-09-10T21:00:26.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1b01448640 1 -- 172.21.15.98:0/3833563155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1afc0b2660 con 0x7f1afc14e070 2024-09-10T21:00:26.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.965+0000 7f1b01448640 1 -- 172.21.15.98:0/3833563155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1afc09fc40 con 0x7f1afc14e070 2024-09-10T21:00:26.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.966+0000 7f1b01448640 1 -- 172.21.15.98:0/3833563155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1afc06d870 con 0x7f1afc14e070 2024-09-10T21:00:26.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.967+0000 7f1af37fe640 1 -- 172.21.15.98:0/3833563155 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1ae402e840 con 0x7f1afc14e070 2024-09-10T21:00:26.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.967+0000 7f1af37fe640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1adc077f70 0x7f1adc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.967+0000 7f1af37fe640 1 -- 172.21.15.98:0/3833563155 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f1ae40ba1e0 con 0x7f1afc14e070 2024-09-10T21:00:26.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.967+0000 7f1af37fe640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.98:6816/2538250805,v1:172.21.15.98:6817/2538250805] conn(0x7f1adc07e4b0 0x7f1adc0808f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:26.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.967+0000 7f1af37fe640 1 -- 172.21.15.98:0/3833563155 --> [v2:172.21.15.98:6816/2538250805,v1:172.21.15.98:6817/2538250805] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1ae4030e90 con 0x7f1adc07e4b0 2024-09-10T21:00:26.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.967+0000 7f1afaffd640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1adc077f70 0x7f1adc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.967+0000 7f1af37fe640 1 -- 172.21.15.98:0/3833563155 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1ae40c1050 con 0x7f1afc14e070 2024-09-10T21:00:26.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.968+0000 7f1afb7fe640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.98:6816/2538250805,v1:172.21.15.98:6817/2538250805] conn(0x7f1adc07e4b0 0x7f1adc0808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:26.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.968+0000 7f1afb7fe640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.98:6816/2538250805,v1:172.21.15.98:6817/2538250805] conn(0x7f1adc07e4b0 0x7f1adc0808f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:26.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.968+0000 7f1afaffd640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1adc077f70 0x7f1adc07a430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1aec003fc0 tx=0x7f1aec006ef0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:26.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.968+0000 7f1af37fe640 1 -- 172.21.15.98:0/3833563155 <== osd.3 v2:172.21.15.98:6816/2538250805 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ae4030e90 con 0x7f1adc07e4b0 2024-09-10T21:00:26.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.981+0000 7f1b01448640 1 -- 172.21.15.98:0/3833563155 --> [v2:172.21.15.98:6816/2538250805,v1:172.21.15.98:6817/2538250805] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1afc0ce940 con 0x7f1adc07e4b0 2024-09-10T21:00:26.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.981+0000 7f1af37fe640 1 -- 172.21.15.98:0/3833563155 <== osd.3 v2:172.21.15.98:6816/2538250805 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1afc0ce940 con 0x7f1adc07e4b0 2024-09-10T21:00:26.985 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-10T21:00:26.985 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:00:26.985 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-10T21:00:26.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.981+0000 7f1b01448640 1 -- 172.21.15.98:0/3833563155 >> [v2:172.21.15.98:6816/2538250805,v1:172.21.15.98:6817/2538250805] conn(0x7f1adc07e4b0 msgr2=0x7f1adc0808f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.981+0000 7f1b01448640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.98:6816/2538250805,v1:172.21.15.98:6817/2538250805] conn(0x7f1adc07e4b0 0x7f1adc0808f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.981+0000 7f1b01448640 1 -- 172.21.15.98:0/3833563155 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1adc077f70 msgr2=0x7f1adc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.981+0000 7f1b01448640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f1adc077f70 0x7f1adc07a430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1aec003fc0 tx=0x7f1aec006ef0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.981+0000 7f1b01448640 1 -- 172.21.15.98:0/3833563155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1afc14e070 msgr2=0x7f1afc14e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:26.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.981+0000 7f1b01448640 1 --2- 172.21.15.98:0/3833563155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1afc14e070 0x7f1afc14e4d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400c320 tx=0x7f1ae4001cf0 comp rx=0 tx=0).stop 2024-09-10T21:00:26.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.981+0000 7f1afb7fe640 1 -- 172.21.15.98:0/3833563155 reap_dead start 2024-09-10T21:00:26.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.981+0000 7f1b01448640 1 -- 172.21.15.98:0/3833563155 shutdown_connections 2024-09-10T21:00:26.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.981+0000 7f1b01448640 1 -- 172.21.15.98:0/3833563155 >> 172.21.15.98:0/3833563155 conn(0x7f1afc09d170 msgr2=0x7f1afc0b5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:26.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.981+0000 7f1b01448640 1 -- 172.21.15.98:0/3833563155 shutdown_connections 2024-09-10T21:00:26.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:26.981+0000 7f1b01448640 1 -- 172.21.15.98:0/3833563155 wait complete. 2024-09-10T21:00:26.996 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-10T21:00:27.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.093+0000 7fa56755d640 1 -- 172.21.15.98:0/2617248521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa560073780 msgr2=0x7fa560073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.093+0000 7fa56755d640 1 --2- 172.21.15.98:0/2617248521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa560073780 0x7fa560073b60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009900 tx=0x7fa55c02a560 comp rx=0 tx=0).stop 2024-09-10T21:00:27.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.093+0000 7fa56755d640 1 -- 172.21.15.98:0/2617248521 shutdown_connections 2024-09-10T21:00:27.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.093+0000 7fa56755d640 1 --2- 172.21.15.98:0/2617248521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa56015c470 0x7fa56015c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.093+0000 7fa56755d640 1 --2- 172.21.15.98:0/2617248521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa560073780 0x7fa560073b60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.093+0000 7fa56755d640 1 -- 172.21.15.98:0/2617248521 >> 172.21.15.98:0/2617248521 conn(0x7fa5600b1760 msgr2=0x7fa5600b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:27.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.093+0000 7fa56755d640 1 -- 172.21.15.98:0/2617248521 shutdown_connections 2024-09-10T21:00:27.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.093+0000 7fa56755d640 1 -- 172.21.15.98:0/2617248521 wait complete. 2024-09-10T21:00:27.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.094+0000 7fa56755d640 1 Processor -- start 2024-09-10T21:00:27.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.094+0000 7fa56755d640 1 -- start start 2024-09-10T21:00:27.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa56755d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa560072560 0x7fa5601f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa56755d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa560073780 0x7fa56013dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa56755d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa56015c470 0x7fa560144840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa56755d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa56015cf70 con 0x7fa560073780 2024-09-10T21:00:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa56755d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa56015cdf0 con 0x7fa56015c470 2024-09-10T21:00:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa56755d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa560070150 con 0x7fa560072560 2024-09-10T21:00:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa564af8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa560073780 0x7fa56013dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa564af8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa560073780 0x7fa56013dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42044/0 (socket says 172.21.15.98:42044) 2024-09-10T21:00:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa564af8640 1 -- 172.21.15.98:0/1375676645 learned_addr learned my addr 172.21.15.98:0/1375676645 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa564af8640 1 -- 172.21.15.98:0/1375676645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa560072560 msgr2=0x7fa5601f3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa5652f9640 1 --2- 172.21.15.98:0/1375676645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa560072560 0x7fa5601f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa565afa640 1 --2- 172.21.15.98:0/1375676645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa56015c470 0x7fa560144840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa564af8640 1 --2- 172.21.15.98:0/1375676645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa560072560 0x7fa5601f3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa564af8640 1 -- 172.21.15.98:0/1375676645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa56015c470 msgr2=0x7fa560144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa564af8640 1 --2- 172.21.15.98:0/1375676645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa56015c470 0x7fa560144840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa564af8640 1 -- 172.21.15.98:0/1375676645 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa560143500 con 0x7fa560073780 2024-09-10T21:00:27.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa564af8640 1 --2- 172.21.15.98:0/1375676645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa560073780 0x7fa56013dc10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c970 tx=0x7fa55000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:27.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.095+0000 7fa555ffb640 1 -- 172.21.15.98:0/1375676645 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550007bb0 con 0x7fa560073780 2024-09-10T21:00:27.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.096+0000 7fa555ffb640 1 -- 172.21.15.98:0/1375676645 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa550007e40 con 0x7fa560073780 2024-09-10T21:00:27.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.096+0000 7fa56755d640 1 -- 172.21.15.98:0/1375676645 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5601f3bd0 con 0x7fa560073780 2024-09-10T21:00:27.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.096+0000 7fa555ffb640 1 -- 172.21.15.98:0/1375676645 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55000f690 con 0x7fa560073780 2024-09-10T21:00:27.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.096+0000 7fa56755d640 1 -- 172.21.15.98:0/1375676645 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa560143a80 con 0x7fa560073780 2024-09-10T21:00:27.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.097+0000 7fa56755d640 1 -- 172.21.15.98:0/1375676645 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5600731e0 con 0x7fa560073780 2024-09-10T21:00:27.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.097+0000 7fa555ffb640 1 -- 172.21.15.98:0/1375676645 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa550020030 con 0x7fa560073780 2024-09-10T21:00:27.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.098+0000 7fa555ffb640 1 --2- 172.21.15.98:0/1375676645 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa544077f70 0x7fa54407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.098+0000 7fa555ffb640 1 -- 172.21.15.98:0/1375676645 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7fa55009a110 con 0x7fa560073780 2024-09-10T21:00:27.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.098+0000 7fa5652f9640 1 --2- 172.21.15.98:0/1375676645 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa544077f70 0x7fa54407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.098+0000 7fa5652f9640 1 --2- 172.21.15.98:0/1375676645 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa544077f70 0x7fa54407a430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa55c033000 tx=0x7fa55c004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:27.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.102+0000 7fa555ffb640 1 -- 172.21.15.98:0/1375676645 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa550014030 con 0x7fa560073780 2024-09-10T21:00:27.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.277+0000 7fa56755d640 1 -- 172.21.15.98:0/1375676645 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fa5600cd6d0 con 0x7fa544077f70 2024-09-10T21:00:27.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.277+0000 7fa555ffb640 1 -- 172.21.15.98:0/1375676645 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa5600cd6d0 con 0x7fa544077f70 2024-09-10T21:00:27.281 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 3 to deep-scrub 2024-09-10T21:00:27.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.279+0000 7fa56755d640 1 -- 172.21.15.98:0/1375676645 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa544077f70 msgr2=0x7fa54407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.279+0000 7fa56755d640 1 --2- 172.21.15.98:0/1375676645 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa544077f70 0x7fa54407a430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa55c033000 tx=0x7fa55c004730 comp rx=0 tx=0).stop 2024-09-10T21:00:27.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.279+0000 7fa56755d640 1 -- 172.21.15.98:0/1375676645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa560073780 msgr2=0x7fa56013dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.279+0000 7fa56755d640 1 --2- 172.21.15.98:0/1375676645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa560073780 0x7fa56013dc10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c970 tx=0x7fa55000ce30 comp rx=0 tx=0).stop 2024-09-10T21:00:27.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.279+0000 7fa56755d640 1 -- 172.21.15.98:0/1375676645 shutdown_connections 2024-09-10T21:00:27.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.279+0000 7fa56755d640 1 --2- 172.21.15.98:0/1375676645 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fa544077f70 0x7fa54407a430 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.279+0000 7fa56755d640 1 --2- 172.21.15.98:0/1375676645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa56015c470 0x7fa560144840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.279+0000 7fa56755d640 1 --2- 172.21.15.98:0/1375676645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa560073780 0x7fa56013dc10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.279+0000 7fa56755d640 1 --2- 172.21.15.98:0/1375676645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa560072560 0x7fa5601f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.279+0000 7fa56755d640 1 -- 172.21.15.98:0/1375676645 >> 172.21.15.98:0/1375676645 conn(0x7fa5600b1760 msgr2=0x7fa5600cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:27.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.279+0000 7fa56755d640 1 -- 172.21.15.98:0/1375676645 shutdown_connections 2024-09-10T21:00:27.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.279+0000 7fa56755d640 1 -- 172.21.15.98:0/1375676645 wait complete. 2024-09-10T21:00:27.294 INFO:tasks.ceph:Scrubbing osd.4 2024-09-10T21:00:27.294 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:00:27.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.396+0000 7f701adec640 1 -- 172.21.15.98:0/1719812273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f701415a940 msgr2=0x7f70140725d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.396+0000 7f701adec640 1 --2- 172.21.15.98:0/1719812273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f701415a940 0x7f70140725d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7008005fd0 tx=0x7f700802a8b0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.396+0000 7f701adec640 1 -- 172.21.15.98:0/1719812273 shutdown_connections 2024-09-10T21:00:27.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.396+0000 7f701adec640 1 --2- 172.21.15.98:0/1719812273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f701415a940 0x7f70140725d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.396+0000 7f701adec640 1 -- 172.21.15.98:0/1719812273 >> 172.21.15.98:0/1719812273 conn(0x7f70140b1760 msgr2=0x7f70140b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:27.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.396+0000 7f701adec640 1 -- 172.21.15.98:0/1719812273 shutdown_connections 2024-09-10T21:00:27.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.396+0000 7f701adec640 1 -- 172.21.15.98:0/1719812273 wait complete. 2024-09-10T21:00:27.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.397+0000 7f701adec640 1 Processor -- start 2024-09-10T21:00:27.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.397+0000 7f701adec640 1 -- start start 2024-09-10T21:00:27.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f701adec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70140b3f30 0x7f701414f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f701adec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7014151530 0x7f70141f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f701adec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f701415a940 0x7f70140aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f701adec640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f701415ae40 con 0x7f7014151530 2024-09-10T21:00:27.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f701adec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f70140b3a40 con 0x7f70140b3f30 2024-09-10T21:00:27.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f701adec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f70140b4380 con 0x7f701415a940 2024-09-10T21:00:27.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f7013fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7014151530 0x7f70141f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f7013fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7014151530 0x7f70141f4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42060/0 (socket says 172.21.15.98:42060) 2024-09-10T21:00:27.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f7013fff640 1 -- 172.21.15.98:0/4031064832 learned_addr learned my addr 172.21.15.98:0/4031064832 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:27.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f7019389640 1 --2- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f701415a940 0x7f70140aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f7013fff640 1 -- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f701415a940 msgr2=0x7f70140aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f7013fff640 1 --2- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f701415a940 0x7f70140aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f7018b88640 1 --2- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70140b3f30 0x7f701414f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f7013fff640 1 -- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70140b3f30 msgr2=0x7f701414f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f7013fff640 1 --2- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70140b3f30 0x7f701414f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f7013fff640 1 -- 172.21.15.98:0/4031064832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70140ae620 con 0x7f7014151530 2024-09-10T21:00:27.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f7013fff640 1 --2- 172.21.15.98:0/4031064832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7014151530 0x7f70141f4f70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7008000f80 tx=0x7f7008003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:27.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f70117fa640 1 -- 172.21.15.98:0/4031064832 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008031da0 con 0x7f7014151530 2024-09-10T21:00:27.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f70117fa640 1 -- 172.21.15.98:0/4031064832 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7008003e10 con 0x7f7014151530 2024-09-10T21:00:27.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.398+0000 7f701adec640 1 -- 172.21.15.98:0/4031064832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70140ae060 con 0x7f7014151530 2024-09-10T21:00:27.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.399+0000 7f70117fa640 1 -- 172.21.15.98:0/4031064832 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008033330 con 0x7f7014151530 2024-09-10T21:00:27.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.399+0000 7f701adec640 1 -- 172.21.15.98:0/4031064832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70141f4770 con 0x7f7014151530 2024-09-10T21:00:27.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.399+0000 7f701adec640 1 -- 172.21.15.98:0/4031064832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f70140b4800 con 0x7f7014151530 2024-09-10T21:00:27.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.400+0000 7f70117fa640 1 -- 172.21.15.98:0/4031064832 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f700804b030 con 0x7f7014151530 2024-09-10T21:00:27.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.401+0000 7f70117fa640 1 --2- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7000077f70 0x7f700007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.401+0000 7f70117fa640 1 -- 172.21.15.98:0/4031064832 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f70080bad50 con 0x7f7014151530 2024-09-10T21:00:27.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.401+0000 7f70117fa640 1 --2- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:6826/2783440087,v1:172.21.15.190:6827/2783440087] conn(0x7f700007e4b0 0x7f70000808f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.401+0000 7f70117fa640 1 -- 172.21.15.98:0/4031064832 --> [v2:172.21.15.190:6826/2783440087,v1:172.21.15.190:6827/2783440087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7008004850 con 0x7f700007e4b0 2024-09-10T21:00:27.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.401+0000 7f70117fa640 1 -- 172.21.15.98:0/4031064832 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f70080bd450 con 0x7f7014151530 2024-09-10T21:00:27.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.401+0000 7f7018b88640 1 --2- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7000077f70 0x7f700007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.401+0000 7f7019389640 1 --2- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:6826/2783440087,v1:172.21.15.190:6827/2783440087] conn(0x7f700007e4b0 0x7f70000808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.401+0000 7f7018b88640 1 --2- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7000077f70 0x7f700007a430 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7014152a40 tx=0x7f7004009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:27.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.402+0000 7f7019389640 1 --2- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:6826/2783440087,v1:172.21.15.190:6827/2783440087] conn(0x7f700007e4b0 0x7f70000808f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:27.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.402+0000 7f70117fa640 1 -- 172.21.15.98:0/4031064832 <== osd.4 v2:172.21.15.190:6826/2783440087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7008004850 con 0x7f700007e4b0 2024-09-10T21:00:27.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.415+0000 7f701adec640 1 -- 172.21.15.98:0/4031064832 --> [v2:172.21.15.190:6826/2783440087,v1:172.21.15.190:6827/2783440087] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f70140af9b0 con 0x7f700007e4b0 2024-09-10T21:00:27.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.415+0000 7f70117fa640 1 -- 172.21.15.98:0/4031064832 <== osd.4 v2:172.21.15.190:6826/2783440087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f70140af9b0 con 0x7f700007e4b0 2024-09-10T21:00:27.419 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-10T21:00:27.419 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:00:27.419 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-10T21:00:27.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.415+0000 7f701adec640 1 -- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:6826/2783440087,v1:172.21.15.190:6827/2783440087] conn(0x7f700007e4b0 msgr2=0x7f70000808f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.415+0000 7f701adec640 1 --2- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:6826/2783440087,v1:172.21.15.190:6827/2783440087] conn(0x7f700007e4b0 0x7f70000808f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.415+0000 7f701adec640 1 -- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7000077f70 msgr2=0x7f700007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.415+0000 7f701adec640 1 --2- 172.21.15.98:0/4031064832 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7000077f70 0x7f700007a430 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7014152a40 tx=0x7f7004009040 comp rx=0 tx=0).stop 2024-09-10T21:00:27.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.415+0000 7f701adec640 1 -- 172.21.15.98:0/4031064832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7014151530 msgr2=0x7f70141f4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.415+0000 7f701adec640 1 --2- 172.21.15.98:0/4031064832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7014151530 0x7f70141f4f70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7008000f80 tx=0x7f7008003c20 comp rx=0 tx=0).stop 2024-09-10T21:00:27.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.415+0000 7f7019389640 1 -- 172.21.15.98:0/4031064832 reap_dead start 2024-09-10T21:00:27.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.416+0000 7f701adec640 1 -- 172.21.15.98:0/4031064832 shutdown_connections 2024-09-10T21:00:27.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.416+0000 7f701adec640 1 -- 172.21.15.98:0/4031064832 >> 172.21.15.98:0/4031064832 conn(0x7f70140b1760 msgr2=0x7f70140702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:27.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.416+0000 7f701adec640 1 -- 172.21.15.98:0/4031064832 shutdown_connections 2024-09-10T21:00:27.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.416+0000 7f701adec640 1 -- 172.21.15.98:0/4031064832 wait complete. 2024-09-10T21:00:27.429 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-10T21:00:27.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.527+0000 7f2b65b94640 1 -- 172.21.15.98:0/2497706464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b600d2bf0 msgr2=0x7f2b60168e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.527+0000 7f2b65b94640 1 --2- 172.21.15.98:0/2497706464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b600d2bf0 0x7f2b60168e00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009be0 tx=0x7f2b5802a840 comp rx=0 tx=0).stop 2024-09-10T21:00:27.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.527+0000 7f2b65b94640 1 -- 172.21.15.98:0/2497706464 shutdown_connections 2024-09-10T21:00:27.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.527+0000 7f2b65b94640 1 --2- 172.21.15.98:0/2497706464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b600d2bf0 0x7f2b60168e00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.527+0000 7f2b65b94640 1 -- 172.21.15.98:0/2497706464 >> 172.21.15.98:0/2497706464 conn(0x7f2b600d3f70 msgr2=0x7f2b600d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:27.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.527+0000 7f2b65b94640 1 -- 172.21.15.98:0/2497706464 shutdown_connections 2024-09-10T21:00:27.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.527+0000 7f2b65b94640 1 -- 172.21.15.98:0/2497706464 wait complete. 2024-09-10T21:00:27.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.528+0000 7f2b65b94640 1 Processor -- start 2024-09-10T21:00:27.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.528+0000 7f2b65b94640 1 -- start start 2024-09-10T21:00:27.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.529+0000 7f2b65b94640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b600d2bf0 0x7f2b6015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.529+0000 7f2b65b94640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b600d4af0 0x7f2b601596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.529+0000 7f2b65b94640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b6015e060 0x7f2b60152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.529+0000 7f2b65b94640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2b601694c0 con 0x7f2b6015e060 2024-09-10T21:00:27.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.529+0000 7f2b65b94640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2b60169340 con 0x7f2b600d2bf0 2024-09-10T21:00:27.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.529+0000 7f2b65b94640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2b60169640 con 0x7f2b600d4af0 2024-09-10T21:00:27.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.529+0000 7f2b5ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b6015e060 0x7f2b60152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.529+0000 7f2b5ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b6015e060 0x7f2b60152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42080/0 (socket says 172.21.15.98:42080) 2024-09-10T21:00:27.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.529+0000 7f2b5ffff640 1 -- 172.21.15.98:0/359750541 learned_addr learned my addr 172.21.15.98:0/359750541 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:27.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b56ffd640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b600d4af0 0x7f2b601596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b5f7fe640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b600d2bf0 0x7f2b6015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b5ffff640 1 -- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b600d4af0 msgr2=0x7f2b601596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b5ffff640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b600d4af0 0x7f2b601596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b5ffff640 1 -- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b600d2bf0 msgr2=0x7f2b6015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b5ffff640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b600d2bf0 0x7f2b6015e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b5ffff640 1 -- 172.21.15.98:0/359750541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6015ecd0 con 0x7f2b6015e060 2024-09-10T21:00:27.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b56ffd640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b600d4af0 0x7f2b601596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:27.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b5f7fe640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b600d2bf0 0x7f2b6015e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:27.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b5ffff640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b6015e060 0x7f2b60152ea0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2b58035040 tx=0x7f2b58033e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:27.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b5cff9640 1 -- 172.21.15.98:0/359750541 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58042020 con 0x7f2b6015e060 2024-09-10T21:00:27.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b5cff9640 1 -- 172.21.15.98:0/359750541 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b58004330 con 0x7f2b6015e060 2024-09-10T21:00:27.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b5cff9640 1 -- 172.21.15.98:0/359750541 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58037040 con 0x7f2b6015e060 2024-09-10T21:00:27.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b65b94640 1 -- 172.21.15.98:0/359750541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6015efb0 con 0x7f2b6015e060 2024-09-10T21:00:27.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.530+0000 7f2b65b94640 1 -- 172.21.15.98:0/359750541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6015f570 con 0x7f2b6015e060 2024-09-10T21:00:27.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.531+0000 7f2b65b94640 1 -- 172.21.15.98:0/359750541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b600d24e0 con 0x7f2b6015e060 2024-09-10T21:00:27.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.532+0000 7f2b5cff9640 1 -- 172.21.15.98:0/359750541 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2b58031070 con 0x7f2b6015e060 2024-09-10T21:00:27.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.532+0000 7f2b5cff9640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2b40077f70 0x7f2b4007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.532+0000 7f2b5cff9640 1 -- 172.21.15.98:0/359750541 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f2b580bac60 con 0x7f2b6015e060 2024-09-10T21:00:27.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.532+0000 7f2b5f7fe640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2b40077f70 0x7f2b4007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.533+0000 7f2b5f7fe640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2b40077f70 0x7f2b4007a430 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2b50000f60 tx=0x7f2b50009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:27.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.536+0000 7f2b5cff9640 1 -- 172.21.15.98:0/359750541 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2b5804c030 con 0x7f2b6015e060 2024-09-10T21:00:27.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.709+0000 7f2b65b94640 1 -- 172.21.15.98:0/359750541 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f2b6008dfa0 con 0x7f2b40077f70 2024-09-10T21:00:27.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.710+0000 7f2b5cff9640 1 -- 172.21.15.98:0/359750541 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2b6008dfa0 con 0x7f2b40077f70 2024-09-10T21:00:27.714 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 4 to deep-scrub 2024-09-10T21:00:27.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.712+0000 7f2b65b94640 1 -- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2b40077f70 msgr2=0x7f2b4007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.712+0000 7f2b65b94640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2b40077f70 0x7f2b4007a430 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2b50000f60 tx=0x7f2b50009040 comp rx=0 tx=0).stop 2024-09-10T21:00:27.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.712+0000 7f2b65b94640 1 -- 172.21.15.98:0/359750541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b6015e060 msgr2=0x7f2b60152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.712+0000 7f2b65b94640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b6015e060 0x7f2b60152ea0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2b58035040 tx=0x7f2b58033e80 comp rx=0 tx=0).stop 2024-09-10T21:00:27.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.712+0000 7f2b65b94640 1 -- 172.21.15.98:0/359750541 shutdown_connections 2024-09-10T21:00:27.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.712+0000 7f2b65b94640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f2b40077f70 0x7f2b4007a430 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.712+0000 7f2b65b94640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b6015e060 0x7f2b60152ea0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.712+0000 7f2b65b94640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b600d4af0 0x7f2b601596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.712+0000 7f2b65b94640 1 --2- 172.21.15.98:0/359750541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b600d2bf0 0x7f2b6015e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.712+0000 7f2b65b94640 1 -- 172.21.15.98:0/359750541 >> 172.21.15.98:0/359750541 conn(0x7f2b600d3f70 msgr2=0x7f2b601621b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:27.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.712+0000 7f2b65b94640 1 -- 172.21.15.98:0/359750541 shutdown_connections 2024-09-10T21:00:27.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.712+0000 7f2b65b94640 1 -- 172.21.15.98:0/359750541 wait complete. 2024-09-10T21:00:27.727 INFO:tasks.ceph:Scrubbing osd.5 2024-09-10T21:00:27.727 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:00:27.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.829+0000 7f82c1fa0640 1 -- 172.21.15.98:0/45494528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82a4003a50 msgr2=0x7f82a4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.829+0000 7f82c1fa0640 1 --2- 172.21.15.98:0/45494528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82a4003a50 0x7f82a4003e70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00c780 tx=0x7f82a4005830 comp rx=0 tx=0).stop 2024-09-10T21:00:27.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.829+0000 7f82c1fa0640 1 -- 172.21.15.98:0/45494528 shutdown_connections 2024-09-10T21:00:27.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.829+0000 7f82c1fa0640 1 --2- 172.21.15.98:0/45494528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82a4003a50 0x7f82a4003e70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.829+0000 7f82c1fa0640 1 -- 172.21.15.98:0/45494528 >> 172.21.15.98:0/45494528 conn(0x7f82bc0b1760 msgr2=0x7f82bc0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:27.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.829+0000 7f82c1fa0640 1 -- 172.21.15.98:0/45494528 shutdown_connections 2024-09-10T21:00:27.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.829+0000 7f82c1fa0640 1 -- 172.21.15.98:0/45494528 wait complete. 2024-09-10T21:00:27.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.830+0000 7f82c1fa0640 1 Processor -- start 2024-09-10T21:00:27.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.831+0000 7f82c1fa0640 1 -- start start 2024-09-10T21:00:27.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.831+0000 7f82c1fa0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a4003a50 0x7f82bc14bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.831+0000 7f82c1fa0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82bc14b2c0 0x7f82bc14a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.831+0000 7f82c1fa0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82bc152e10 0x7f82bc1532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.831+0000 7f82c1fa0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc0725d0 con 0x7f82bc14b2c0 2024-09-10T21:00:27.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.831+0000 7f82c1fa0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc0b4c30 con 0x7f82bc152e10 2024-09-10T21:00:27.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.831+0000 7f82c1fa0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f82bc072750 con 0x7f82a4003a50 2024-09-10T21:00:27.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.831+0000 7f82baffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82bc14b2c0 0x7f82bc14a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.831+0000 7f82baffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82bc14b2c0 0x7f82bc14a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42098/0 (socket says 172.21.15.98:42098) 2024-09-10T21:00:27.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.831+0000 7f82baffd640 1 -- 172.21.15.98:0/3623026240 learned_addr learned my addr 172.21.15.98:0/3623026240 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:27.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82bb7fe640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a4003a50 0x7f82bc14bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82baffd640 1 -- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a4003a50 msgr2=0x7f82bc14bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82bbfff640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82bc152e10 0x7f82bc1532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82baffd640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a4003a50 0x7f82bc14bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82baffd640 1 -- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82bc152e10 msgr2=0x7f82bc1532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82baffd640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82bc152e10 0x7f82bc1532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82baffd640 1 -- 172.21.15.98:0/3623026240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc1f3e50 con 0x7f82bc14b2c0 2024-09-10T21:00:27.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82bb7fe640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a4003a50 0x7f82bc14bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:27.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82bbfff640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82bc152e10 0x7f82bc1532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:27.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82baffd640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82bc14b2c0 0x7f82bc14a5c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f82b0006690 tx=0x7f82b00016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:27.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82a3fff640 1 -- 172.21.15.98:0/3623026240 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b0015020 con 0x7f82bc14b2c0 2024-09-10T21:00:27.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82a3fff640 1 -- 172.21.15.98:0/3623026240 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82b0011900 con 0x7f82bc14b2c0 2024-09-10T21:00:27.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82a3fff640 1 -- 172.21.15.98:0/3623026240 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b0011d50 con 0x7f82bc14b2c0 2024-09-10T21:00:27.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82c1fa0640 1 -- 172.21.15.98:0/3623026240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82bc1f4130 con 0x7f82bc14b2c0 2024-09-10T21:00:27.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.832+0000 7f82c1fa0640 1 -- 172.21.15.98:0/3623026240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82bc1f46f0 con 0x7f82bc14b2c0 2024-09-10T21:00:27.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.833+0000 7f82c1fa0640 1 -- 172.21.15.98:0/3623026240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f82bc14e670 con 0x7f82bc14b2c0 2024-09-10T21:00:27.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.834+0000 7f82a3fff640 1 -- 172.21.15.98:0/3623026240 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f82b001a400 con 0x7f82bc14b2c0 2024-09-10T21:00:27.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.834+0000 7f82a3fff640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f82a8077fa0 0x7f82a807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.834+0000 7f82a3fff640 1 -- 172.21.15.98:0/3623026240 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f82b009aa30 con 0x7f82bc14b2c0 2024-09-10T21:00:27.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.834+0000 7f82a3fff640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:6802/904517515,v1:172.21.15.190:6803/904517515] conn(0x7f82a807e4e0 0x7f82a8080920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.834+0000 7f82a3fff640 1 -- 172.21.15.98:0/3623026240 --> [v2:172.21.15.190:6802/904517515,v1:172.21.15.190:6803/904517515] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f82b000ee50 con 0x7f82a807e4e0 2024-09-10T21:00:27.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.834+0000 7f82a3fff640 1 -- 172.21.15.98:0/3623026240 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f82b009f2c0 con 0x7f82bc14b2c0 2024-09-10T21:00:27.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.834+0000 7f82bb7fe640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f82a8077fa0 0x7f82a807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.835+0000 7f82bbfff640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:6802/904517515,v1:172.21.15.190:6803/904517515] conn(0x7f82a807e4e0 0x7f82a8080920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.835+0000 7f82bb7fe640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f82a8077fa0 0x7f82a807a460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f82ac033000 tx=0x7f82ac02d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:27.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.835+0000 7f82bbfff640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:6802/904517515,v1:172.21.15.190:6803/904517515] conn(0x7f82a807e4e0 0x7f82a8080920 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:27.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.836+0000 7f82a3fff640 1 -- 172.21.15.98:0/3623026240 <== osd.5 v2:172.21.15.190:6802/904517515 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f82b000ee50 con 0x7f82a807e4e0 2024-09-10T21:00:27.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.848+0000 7f82c1fa0640 1 -- 172.21.15.98:0/3623026240 --> [v2:172.21.15.190:6802/904517515,v1:172.21.15.190:6803/904517515] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f82bc0cc570 con 0x7f82a807e4e0 2024-09-10T21:00:27.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.849+0000 7f82a3fff640 1 -- 172.21.15.98:0/3623026240 <== osd.5 v2:172.21.15.190:6802/904517515 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f82bc0cc570 con 0x7f82a807e4e0 2024-09-10T21:00:27.853 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-10T21:00:27.853 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:00:27.853 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-10T21:00:27.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.849+0000 7f82c1fa0640 1 -- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:6802/904517515,v1:172.21.15.190:6803/904517515] conn(0x7f82a807e4e0 msgr2=0x7f82a8080920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.849+0000 7f82c1fa0640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:6802/904517515,v1:172.21.15.190:6803/904517515] conn(0x7f82a807e4e0 0x7f82a8080920 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.849+0000 7f82c1fa0640 1 -- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f82a8077fa0 msgr2=0x7f82a807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.849+0000 7f82c1fa0640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f82a8077fa0 0x7f82a807a460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f82ac033000 tx=0x7f82ac02d040 comp rx=0 tx=0).stop 2024-09-10T21:00:27.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.849+0000 7f82c1fa0640 1 -- 172.21.15.98:0/3623026240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82bc14b2c0 msgr2=0x7f82bc14a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.849+0000 7f82c1fa0640 1 --2- 172.21.15.98:0/3623026240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f82bc14b2c0 0x7f82bc14a5c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f82b0006690 tx=0x7f82b00016f0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.849+0000 7f82bbfff640 1 -- 172.21.15.98:0/3623026240 reap_dead start 2024-09-10T21:00:27.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.849+0000 7f82c1fa0640 1 -- 172.21.15.98:0/3623026240 shutdown_connections 2024-09-10T21:00:27.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.849+0000 7f82c1fa0640 1 -- 172.21.15.98:0/3623026240 >> 172.21.15.98:0/3623026240 conn(0x7f82bc0b1760 msgr2=0x7f82bc0cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:27.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.849+0000 7f82c1fa0640 1 -- 172.21.15.98:0/3623026240 shutdown_connections 2024-09-10T21:00:27.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.849+0000 7f82c1fa0640 1 -- 172.21.15.98:0/3623026240 wait complete. 2024-09-10T21:00:27.863 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-10T21:00:27.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.961+0000 7f7df8c41640 1 -- 172.21.15.98:0/853134331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df40d2a20 msgr2=0x7f7df40cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.961+0000 7f7df8c41640 1 --2- 172.21.15.98:0/853134331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df40d2a20 0x7f7df40cea40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7de4005fd0 tx=0x7f7de402a870 comp rx=0 tx=0).stop 2024-09-10T21:00:27.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.961+0000 7f7df8c41640 1 -- 172.21.15.98:0/853134331 shutdown_connections 2024-09-10T21:00:27.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.961+0000 7f7df8c41640 1 --2- 172.21.15.98:0/853134331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df40d2a20 0x7f7df40cea40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.961+0000 7f7df8c41640 1 --2- 172.21.15.98:0/853134331 >> v2:172.21.15.190:3300/0 conn(0x7f7df40d4af0 0x7f7df40d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.961+0000 7f7df8c41640 1 -- 172.21.15.98:0/853134331 >> 172.21.15.98:0/853134331 conn(0x7f7df40d3f70 msgr2=0x7f7df40d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:27.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.961+0000 7f7df8c41640 1 -- 172.21.15.98:0/853134331 shutdown_connections 2024-09-10T21:00:27.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.961+0000 7f7df8c41640 1 -- 172.21.15.98:0/853134331 wait complete. 2024-09-10T21:00:27.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.963+0000 7f7df8c41640 1 Processor -- start 2024-09-10T21:00:27.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.963+0000 7f7df8c41640 1 -- start start 2024-09-10T21:00:27.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.963+0000 7f7df8c41640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7df40d2a20 0x7f7df40b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.963+0000 7f7df8c41640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df40d4af0 0x7f7df4155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.963+0000 7f7df8c41640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7df415e060 0x7f7df4160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.963+0000 7f7df8c41640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7df40cef80 con 0x7f7df40d4af0 2024-09-10T21:00:27.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.963+0000 7f7df8c41640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7df40d2f20 con 0x7f7df415e060 2024-09-10T21:00:27.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.963+0000 7f7df8c41640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7df40cf100 con 0x7f7df40d2a20 2024-09-10T21:00:27.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7df1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df40d4af0 0x7f7df4155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7df1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df40d4af0 0x7f7df4155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42112/0 (socket says 172.21.15.98:42112) 2024-09-10T21:00:27.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7df1d9b640 1 -- 172.21.15.98:0/2438643614 learned_addr learned my addr 172.21.15.98:0/2438643614 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:27.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7df2d9d640 1 --2- 172.21.15.98:0/2438643614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7df415e060 0x7f7df4160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7df259c640 1 --2- 172.21.15.98:0/2438643614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7df40d2a20 0x7f7df40b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7df1d9b640 1 -- 172.21.15.98:0/2438643614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7df40d2a20 msgr2=0x7f7df40b7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7df1d9b640 1 --2- 172.21.15.98:0/2438643614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7df40d2a20 0x7f7df40b7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7df1d9b640 1 -- 172.21.15.98:0/2438643614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7df415e060 msgr2=0x7f7df4160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:27.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7df1d9b640 1 --2- 172.21.15.98:0/2438643614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7df415e060 0x7f7df4160960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:27.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7df1d9b640 1 -- 172.21.15.98:0/2438643614 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df4159cd0 con 0x7f7df40d4af0 2024-09-10T21:00:27.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7df1d9b640 1 --2- 172.21.15.98:0/2438643614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df40d4af0 0x7f7df4155d70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00d9b0 tx=0x7f7ddc00de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:27.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7deaffd640 1 -- 172.21.15.98:0/2438643614 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc017070 con 0x7f7df40d4af0 2024-09-10T21:00:27.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7deaffd640 1 -- 172.21.15.98:0/2438643614 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ddc009cc0 con 0x7f7df40d4af0 2024-09-10T21:00:27.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7deaffd640 1 -- 172.21.15.98:0/2438643614 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0106c0 con 0x7f7df40d4af0 2024-09-10T21:00:27.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7df8c41640 1 -- 172.21.15.98:0/2438643614 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df4159fb0 con 0x7f7df40d4af0 2024-09-10T21:00:27.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.964+0000 7f7df8c41640 1 -- 172.21.15.98:0/2438643614 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df40c87c0 con 0x7f7df40d4af0 2024-09-10T21:00:27.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.965+0000 7f7df8c41640 1 -- 172.21.15.98:0/2438643614 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df40cea40 con 0x7f7df40d4af0 2024-09-10T21:00:27.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.966+0000 7f7deaffd640 1 -- 172.21.15.98:0/2438643614 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7ddc010860 con 0x7f7df40d4af0 2024-09-10T21:00:27.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.966+0000 7f7deaffd640 1 --2- 172.21.15.98:0/2438643614 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7dd4077f70 0x7f7dd407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:27.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.966+0000 7f7deaffd640 1 -- 172.21.15.98:0/2438643614 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f7ddc09a000 con 0x7f7df40d4af0 2024-09-10T21:00:27.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.966+0000 7f7df259c640 1 --2- 172.21.15.98:0/2438643614 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7dd4077f70 0x7f7dd407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:27.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.967+0000 7f7df259c640 1 --2- 172.21.15.98:0/2438643614 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7dd4077f70 0x7f7dd407a430 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7dec007fd0 tx=0x7f7dec009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:27.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:27.971+0000 7f7deaffd640 1 -- 172.21.15.98:0/2438643614 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7ddc062c20 con 0x7f7df40d4af0 2024-09-10T21:00:28.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.143+0000 7f7df8c41640 1 -- 172.21.15.98:0/2438643614 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f7df4075ee0 con 0x7f7dd4077f70 2024-09-10T21:00:28.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.143+0000 7f7deaffd640 1 -- 172.21.15.98:0/2438643614 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7df4075ee0 con 0x7f7dd4077f70 2024-09-10T21:00:28.147 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 5 to deep-scrub 2024-09-10T21:00:28.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.145+0000 7f7df8c41640 1 -- 172.21.15.98:0/2438643614 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7dd4077f70 msgr2=0x7f7dd407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.145+0000 7f7df8c41640 1 --2- 172.21.15.98:0/2438643614 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7dd4077f70 0x7f7dd407a430 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7dec007fd0 tx=0x7f7dec009040 comp rx=0 tx=0).stop 2024-09-10T21:00:28.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.145+0000 7f7df8c41640 1 -- 172.21.15.98:0/2438643614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df40d4af0 msgr2=0x7f7df4155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.145+0000 7f7df8c41640 1 --2- 172.21.15.98:0/2438643614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df40d4af0 0x7f7df4155d70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00d9b0 tx=0x7f7ddc00de70 comp rx=0 tx=0).stop 2024-09-10T21:00:28.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.145+0000 7f7df8c41640 1 -- 172.21.15.98:0/2438643614 shutdown_connections 2024-09-10T21:00:28.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.145+0000 7f7df8c41640 1 --2- 172.21.15.98:0/2438643614 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7dd4077f70 0x7f7dd407a430 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.145+0000 7f7df8c41640 1 --2- 172.21.15.98:0/2438643614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7df415e060 0x7f7df4160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.145+0000 7f7df8c41640 1 --2- 172.21.15.98:0/2438643614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7df40d4af0 0x7f7df4155d70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.145+0000 7f7df8c41640 1 --2- 172.21.15.98:0/2438643614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7df40d2a20 0x7f7df40b7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.145+0000 7f7df8c41640 1 -- 172.21.15.98:0/2438643614 >> 172.21.15.98:0/2438643614 conn(0x7f7df40d3f70 msgr2=0x7f7df406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:28.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.146+0000 7f7df8c41640 1 -- 172.21.15.98:0/2438643614 shutdown_connections 2024-09-10T21:00:28.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.146+0000 7f7df8c41640 1 -- 172.21.15.98:0/2438643614 wait complete. 2024-09-10T21:00:28.160 INFO:tasks.ceph:Scrubbing osd.6 2024-09-10T21:00:28.160 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:00:28.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.262+0000 7f7464ab8640 1 -- 172.21.15.98:0/2503765568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f746015e530 msgr2=0x7f74600cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.262+0000 7f7464ab8640 1 --2- 172.21.15.98:0/2503765568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f746015e530 0x7f74600cf110 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7458005fd0 tx=0x7f745802a8b0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.262+0000 7f7464ab8640 1 -- 172.21.15.98:0/2503765568 shutdown_connections 2024-09-10T21:00:28.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.262+0000 7f7464ab8640 1 --2- 172.21.15.98:0/2503765568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f746015e530 0x7f74600cf110 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.262+0000 7f7464ab8640 1 -- 172.21.15.98:0/2503765568 >> 172.21.15.98:0/2503765568 conn(0x7f74600b1760 msgr2=0x7f74600b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:28.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.262+0000 7f7464ab8640 1 -- 172.21.15.98:0/2503765568 shutdown_connections 2024-09-10T21:00:28.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.262+0000 7f7464ab8640 1 -- 172.21.15.98:0/2503765568 wait complete. 2024-09-10T21:00:28.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.264+0000 7f7464ab8640 1 Processor -- start 2024-09-10T21:00:28.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.264+0000 7f7464ab8640 1 -- start start 2024-09-10T21:00:28.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.264+0000 7f7464ab8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f746006e2e0 0x7f7460159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.264+0000 7f7464ab8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f746015e150 0x7f746015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.264+0000 7f7464ab8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f746015e530 0x7f746015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.264+0000 7f7464ab8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f74600b4930 con 0x7f746015e530 2024-09-10T21:00:28.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.264+0000 7f7464ab8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f74600b47b0 con 0x7f746015e150 2024-09-10T21:00:28.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.264+0000 7f7464ab8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f74600b4ab0 con 0x7f746006e2e0 2024-09-10T21:00:28.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f745ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f746015e530 0x7f746015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f745ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f746015e530 0x7f746015bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42122/0 (socket says 172.21.15.98:42122) 2024-09-10T21:00:28.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f745ed9d640 1 -- 172.21.15.98:0/950593113 learned_addr learned my addr 172.21.15.98:0/950593113 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:28.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f745e59c640 1 --2- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f746006e2e0 0x7f7460159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f7457fff640 1 --2- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f746015e150 0x7f746015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f745ed9d640 1 -- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f746006e2e0 msgr2=0x7f7460159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f745ed9d640 1 --2- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f746006e2e0 0x7f7460159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f745ed9d640 1 -- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f746015e150 msgr2=0x7f746015b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f745ed9d640 1 --2- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f746015e150 0x7f746015b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f745ed9d640 1 -- 172.21.15.98:0/950593113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7460153760 con 0x7f746015e530 2024-09-10T21:00:28.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f745ed9d640 1 --2- 172.21.15.98:0/950593113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f746015e530 0x7f746015bf00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7458009cd0 tx=0x7f74580039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:28.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f7456ffd640 1 -- 172.21.15.98:0/950593113 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458031990 con 0x7f746015e530 2024-09-10T21:00:28.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f7456ffd640 1 -- 172.21.15.98:0/950593113 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7458031c20 con 0x7f746015e530 2024-09-10T21:00:28.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f7464ab8640 1 -- 172.21.15.98:0/950593113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74601531a0 con 0x7f746015e530 2024-09-10T21:00:28.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f7456ffd640 1 -- 172.21.15.98:0/950593113 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74580330d0 con 0x7f746015e530 2024-09-10T21:00:28.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.265+0000 7f7464ab8640 1 -- 172.21.15.98:0/950593113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74601572f0 con 0x7f746015e530 2024-09-10T21:00:28.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.266+0000 7f7464ab8640 1 -- 172.21.15.98:0/950593113 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7460072740 con 0x7f746015e530 2024-09-10T21:00:28.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.267+0000 7f7456ffd640 1 -- 172.21.15.98:0/950593113 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f745804b030 con 0x7f746015e530 2024-09-10T21:00:28.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.267+0000 7f7456ffd640 1 --2- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7440077f70 0x7f744007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.268+0000 7f7456ffd640 1 -- 172.21.15.98:0/950593113 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f74580bb3e0 con 0x7f746015e530 2024-09-10T21:00:28.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.268+0000 7f7456ffd640 1 --2- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:6810/2106840781,v1:172.21.15.190:6811/2106840781] conn(0x7f744007e4b0 0x7f74400808f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.268+0000 7f7456ffd640 1 -- 172.21.15.98:0/950593113 --> [v2:172.21.15.190:6810/2106840781,v1:172.21.15.190:6811/2106840781] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74580317e0 con 0x7f744007e4b0 2024-09-10T21:00:28.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.268+0000 7f7456ffd640 1 -- 172.21.15.98:0/950593113 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f74580bb870 con 0x7f746015e530 2024-09-10T21:00:28.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.268+0000 7f745e59c640 1 --2- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7440077f70 0x7f744007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.268+0000 7f7457fff640 1 --2- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:6810/2106840781,v1:172.21.15.190:6811/2106840781] conn(0x7f744007e4b0 0x7f74400808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.268+0000 7f745e59c640 1 --2- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7440077f70 0x7f744007a430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f74500017f0 tx=0x7f7450009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:28.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.268+0000 7f7457fff640 1 --2- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:6810/2106840781,v1:172.21.15.190:6811/2106840781] conn(0x7f744007e4b0 0x7f74400808f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:28.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.269+0000 7f7456ffd640 1 -- 172.21.15.98:0/950593113 <== osd.6 v2:172.21.15.190:6810/2106840781 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74580317e0 con 0x7f744007e4b0 2024-09-10T21:00:28.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.281+0000 7f7464ab8640 1 -- 172.21.15.98:0/950593113 --> [v2:172.21.15.190:6810/2106840781,v1:172.21.15.190:6811/2106840781] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f746013ed20 con 0x7f744007e4b0 2024-09-10T21:00:28.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.282+0000 7f7456ffd640 1 -- 172.21.15.98:0/950593113 <== osd.6 v2:172.21.15.190:6810/2106840781 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7448007dd0 con 0x7f744007e4b0 2024-09-10T21:00:28.287 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-10T21:00:28.287 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:00:28.287 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-10T21:00:28.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.282+0000 7f7464ab8640 1 -- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:6810/2106840781,v1:172.21.15.190:6811/2106840781] conn(0x7f744007e4b0 msgr2=0x7f74400808f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.282+0000 7f7464ab8640 1 --2- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:6810/2106840781,v1:172.21.15.190:6811/2106840781] conn(0x7f744007e4b0 0x7f74400808f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.282+0000 7f7464ab8640 1 -- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7440077f70 msgr2=0x7f744007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.282+0000 7f7464ab8640 1 --2- 172.21.15.98:0/950593113 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f7440077f70 0x7f744007a430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f74500017f0 tx=0x7f7450009040 comp rx=0 tx=0).stop 2024-09-10T21:00:28.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.282+0000 7f7464ab8640 1 -- 172.21.15.98:0/950593113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f746015e530 msgr2=0x7f746015bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.282+0000 7f7464ab8640 1 --2- 172.21.15.98:0/950593113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f746015e530 0x7f746015bf00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7458009cd0 tx=0x7f74580039f0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.283+0000 7f745ed9d640 1 -- 172.21.15.98:0/950593113 reap_dead start 2024-09-10T21:00:28.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.283+0000 7f7464ab8640 1 -- 172.21.15.98:0/950593113 shutdown_connections 2024-09-10T21:00:28.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.283+0000 7f7464ab8640 1 -- 172.21.15.98:0/950593113 >> 172.21.15.98:0/950593113 conn(0x7f74600b1760 msgr2=0x7f746014c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:28.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.283+0000 7f7464ab8640 1 -- 172.21.15.98:0/950593113 shutdown_connections 2024-09-10T21:00:28.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.283+0000 7f7464ab8640 1 -- 172.21.15.98:0/950593113 wait complete. 2024-09-10T21:00:28.297 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-10T21:00:28.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.395+0000 7fcb5f19f640 1 -- 172.21.15.98:0/986974204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb580b47d0 msgr2=0x7fcb580d2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.395+0000 7fcb5f19f640 1 --2- 172.21.15.98:0/986974204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb580b47d0 0x7fcb580d2810 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcb50005fd0 tx=0x7fcb5002a540 comp rx=0 tx=0).stop 2024-09-10T21:00:28.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.395+0000 7fcb5f19f640 1 -- 172.21.15.98:0/986974204 shutdown_connections 2024-09-10T21:00:28.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.395+0000 7fcb5f19f640 1 --2- 172.21.15.98:0/986974204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580d18c0 0x7fcb580d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.395+0000 7fcb5f19f640 1 --2- 172.21.15.98:0/986974204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb580b47d0 0x7fcb580d2810 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.395+0000 7fcb5f19f640 1 -- 172.21.15.98:0/986974204 >> 172.21.15.98:0/986974204 conn(0x7fcb5809cea0 msgr2=0x7fcb580d3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:28.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.395+0000 7fcb5f19f640 1 -- 172.21.15.98:0/986974204 shutdown_connections 2024-09-10T21:00:28.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.395+0000 7fcb5f19f640 1 -- 172.21.15.98:0/986974204 wait complete. 2024-09-10T21:00:28.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.396+0000 7fcb5f19f640 1 Processor -- start 2024-09-10T21:00:28.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.396+0000 7fcb5f19f640 1 -- start start 2024-09-10T21:00:28.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5f19f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580b47d0 0x7fcb580ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5f19f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb580d18c0 0x7fcb580cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5f19f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb58162490 0x7fcb581f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5f19f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcb580d0700 con 0x7fcb58162490 2024-09-10T21:00:28.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5f19f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcb580d0580 con 0x7fcb580d18c0 2024-09-10T21:00:28.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5f19f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcb580d0880 con 0x7fcb580b47d0 2024-09-10T21:00:28.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5e99e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb58162490 0x7fcb581f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5e99e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb58162490 0x7fcb581f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42146/0 (socket says 172.21.15.98:42146) 2024-09-10T21:00:28.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5e99e640 1 -- 172.21.15.98:0/3936620882 learned_addr learned my addr 172.21.15.98:0/3936620882 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:28.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5e19d640 1 --2- 172.21.15.98:0/3936620882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580b47d0 0x7fcb580ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5d99c640 1 --2- 172.21.15.98:0/3936620882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb580d18c0 0x7fcb580cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5e99e640 1 -- 172.21.15.98:0/3936620882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580b47d0 msgr2=0x7fcb580ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5e99e640 1 --2- 172.21.15.98:0/3936620882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580b47d0 0x7fcb580ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5e99e640 1 -- 172.21.15.98:0/3936620882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb580d18c0 msgr2=0x7fcb580cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.397+0000 7fcb5e99e640 1 --2- 172.21.15.98:0/3936620882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb580d18c0 0x7fcb580cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.398+0000 7fcb5e99e640 1 -- 172.21.15.98:0/3936620882 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb580cdaa0 con 0x7fcb58162490 2024-09-10T21:00:28.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.398+0000 7fcb5e99e640 1 --2- 172.21.15.98:0/3936620882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb58162490 0x7fcb581f4a00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009230 tx=0x7fcb50003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:28.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.398+0000 7fcb56ffd640 1 -- 172.21.15.98:0/3936620882 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5003f070 con 0x7fcb58162490 2024-09-10T21:00:28.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.398+0000 7fcb56ffd640 1 -- 172.21.15.98:0/3936620882 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb500047a0 con 0x7fcb58162490 2024-09-10T21:00:28.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.398+0000 7fcb56ffd640 1 -- 172.21.15.98:0/3936620882 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50035030 con 0x7fcb58162490 2024-09-10T21:00:28.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.398+0000 7fcb5f19f640 1 -- 172.21.15.98:0/3936620882 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb581f55e0 con 0x7fcb58162490 2024-09-10T21:00:28.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.398+0000 7fcb5f19f640 1 -- 172.21.15.98:0/3936620882 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb580c9c60 con 0x7fcb58162490 2024-09-10T21:00:28.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.399+0000 7fcb5f19f640 1 -- 172.21.15.98:0/3936620882 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb580d0d00 con 0x7fcb58162490 2024-09-10T21:00:28.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.400+0000 7fcb56ffd640 1 -- 172.21.15.98:0/3936620882 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcb50004bf0 con 0x7fcb58162490 2024-09-10T21:00:28.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.400+0000 7fcb56ffd640 1 --2- 172.21.15.98:0/3936620882 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcb44077f70 0x7fcb4407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.400+0000 7fcb56ffd640 1 -- 172.21.15.98:0/3936620882 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7fcb500bacb0 con 0x7fcb58162490 2024-09-10T21:00:28.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.400+0000 7fcb5e19d640 1 --2- 172.21.15.98:0/3936620882 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcb44077f70 0x7fcb4407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.401+0000 7fcb5e19d640 1 --2- 172.21.15.98:0/3936620882 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcb44077f70 0x7fcb4407a430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb48001e20 tx=0x7fcb48009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:28.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.404+0000 7fcb56ffd640 1 -- 172.21.15.98:0/3936620882 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fcb50083280 con 0x7fcb58162490 2024-09-10T21:00:28.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.578+0000 7fcb5f19f640 1 -- 172.21.15.98:0/3936620882 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fcb580b9c70 con 0x7fcb44077f70 2024-09-10T21:00:28.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.579+0000 7fcb56ffd640 1 -- 172.21.15.98:0/3936620882 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcb580b9c70 con 0x7fcb44077f70 2024-09-10T21:00:28.583 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 6 to deep-scrub 2024-09-10T21:00:28.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.581+0000 7fcb5f19f640 1 -- 172.21.15.98:0/3936620882 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcb44077f70 msgr2=0x7fcb4407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.581+0000 7fcb5f19f640 1 --2- 172.21.15.98:0/3936620882 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcb44077f70 0x7fcb4407a430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb48001e20 tx=0x7fcb48009040 comp rx=0 tx=0).stop 2024-09-10T21:00:28.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.581+0000 7fcb5f19f640 1 -- 172.21.15.98:0/3936620882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb58162490 msgr2=0x7fcb581f4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.581+0000 7fcb5f19f640 1 --2- 172.21.15.98:0/3936620882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb58162490 0x7fcb581f4a00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009230 tx=0x7fcb50003890 comp rx=0 tx=0).stop 2024-09-10T21:00:28.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.581+0000 7fcb5f19f640 1 -- 172.21.15.98:0/3936620882 shutdown_connections 2024-09-10T21:00:28.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.581+0000 7fcb5f19f640 1 --2- 172.21.15.98:0/3936620882 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fcb44077f70 0x7fcb4407a430 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.581+0000 7fcb5f19f640 1 --2- 172.21.15.98:0/3936620882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcb58162490 0x7fcb581f4a00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.581+0000 7fcb5f19f640 1 --2- 172.21.15.98:0/3936620882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb580d18c0 0x7fcb580cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.581+0000 7fcb5f19f640 1 --2- 172.21.15.98:0/3936620882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb580b47d0 0x7fcb580ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.581+0000 7fcb5f19f640 1 -- 172.21.15.98:0/3936620882 >> 172.21.15.98:0/3936620882 conn(0x7fcb5809cea0 msgr2=0x7fcb581605e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:28.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.581+0000 7fcb5f19f640 1 -- 172.21.15.98:0/3936620882 shutdown_connections 2024-09-10T21:00:28.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.581+0000 7fcb5f19f640 1 -- 172.21.15.98:0/3936620882 wait complete. 2024-09-10T21:00:28.596 INFO:tasks.ceph:Scrubbing osd.7 2024-09-10T21:00:28.596 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:00:28.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.697+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1638545665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54b0003a50 msgr2=0x7f54b0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.697+0000 7f54cd1f4640 1 --2- 172.21.15.98:0/1638545665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54b0003a50 0x7f54b0003e70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f54b8001050 tx=0x7f54b0005830 comp rx=0 tx=0).stop 2024-09-10T21:00:28.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.697+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1638545665 shutdown_connections 2024-09-10T21:00:28.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.697+0000 7f54cd1f4640 1 --2- 172.21.15.98:0/1638545665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54b0003a50 0x7f54b0003e70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.697+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1638545665 >> 172.21.15.98:0/1638545665 conn(0x7f54c80b1760 msgr2=0x7f54c80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:28.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.697+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1638545665 shutdown_connections 2024-09-10T21:00:28.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.697+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1638545665 wait complete. 2024-09-10T21:00:28.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.698+0000 7f54cd1f4640 1 Processor -- start 2024-09-10T21:00:28.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.698+0000 7f54cd1f4640 1 -- start start 2024-09-10T21:00:28.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.699+0000 7f54cd1f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b0003a50 0x7f54c81f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.699+0000 7f54cd1f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c80ad890 0x7f54c80cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.699+0000 7f54cd1f4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54c813f430 0x7f54c813f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.699+0000 7f54cd1f4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f54c80cec80 con 0x7f54c813f430 2024-09-10T21:00:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.699+0000 7f54cd1f4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f54c80ceb00 con 0x7f54b0003a50 2024-09-10T21:00:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.699+0000 7f54cd1f4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f54c80cee00 con 0x7f54c80ad890 2024-09-10T21:00:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.699+0000 7f54c759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54c813f430 0x7f54c813f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.699+0000 7f54c759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54c813f430 0x7f54c813f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42174/0 (socket says 172.21.15.98:42174) 2024-09-10T21:00:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.699+0000 7f54c759e640 1 -- 172.21.15.98:0/1840381408 learned_addr learned my addr 172.21.15.98:0/1840381408 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.699+0000 7f54c659c640 1 --2- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c80ad890 0x7f54c80cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.699+0000 7f54c6d9d640 1 --2- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b0003a50 0x7f54c81f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.700+0000 7f54c759e640 1 -- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c80ad890 msgr2=0x7f54c80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.700+0000 7f54c759e640 1 --2- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54c80ad890 0x7f54c80cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.700+0000 7f54c759e640 1 -- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b0003a50 msgr2=0x7f54c81f5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.700+0000 7f54c759e640 1 --2- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54b0003a50 0x7f54c81f5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.700+0000 7f54c759e640 1 -- 172.21.15.98:0/1840381408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c80add90 con 0x7f54c813f430 2024-09-10T21:00:28.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.700+0000 7f54c759e640 1 --2- 172.21.15.98:0/1840381408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54c813f430 0x7f54c813f8a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f54c0005fd0 tx=0x7f54c0010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:28.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.700+0000 7f54af7fe640 1 -- 172.21.15.98:0/1840381408 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0003970 con 0x7f54c813f430 2024-09-10T21:00:28.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.700+0000 7f54af7fe640 1 -- 172.21.15.98:0/1840381408 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54c0015070 con 0x7f54c813f430 2024-09-10T21:00:28.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.700+0000 7f54af7fe640 1 -- 172.21.15.98:0/1840381408 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0004030 con 0x7f54c813f430 2024-09-10T21:00:28.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.700+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1840381408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c80ae070 con 0x7f54c813f430 2024-09-10T21:00:28.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.700+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1840381408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c80ae630 con 0x7f54c813f430 2024-09-10T21:00:28.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.701+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1840381408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f54c813fde0 con 0x7f54c813f430 2024-09-10T21:00:28.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.701+0000 7f54af7fe640 1 -- 172.21.15.98:0/1840381408 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f54c00041d0 con 0x7f54c813f430 2024-09-10T21:00:28.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.702+0000 7f54af7fe640 1 --2- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f54b4077f70 0x7f54b407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.702+0000 7f54af7fe640 1 -- 172.21.15.98:0/1840381408 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f54c009a600 con 0x7f54c813f430 2024-09-10T21:00:28.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.702+0000 7f54af7fe640 1 --2- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:6818/3616946293,v1:172.21.15.190:6819/3616946293] conn(0x7f54b407e4b0 0x7f54b40808f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.702+0000 7f54af7fe640 1 -- 172.21.15.98:0/1840381408 --> [v2:172.21.15.190:6818/3616946293,v1:172.21.15.190:6819/3616946293] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54c0018e50 con 0x7f54b407e4b0 2024-09-10T21:00:28.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.702+0000 7f54af7fe640 1 -- 172.21.15.98:0/1840381408 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f54c00a1050 con 0x7f54c813f430 2024-09-10T21:00:28.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.702+0000 7f54c6d9d640 1 --2- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f54b4077f70 0x7f54b407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.702+0000 7f54c659c640 1 --2- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:6818/3616946293,v1:172.21.15.190:6819/3616946293] conn(0x7f54b407e4b0 0x7f54b40808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.703+0000 7f54c6d9d640 1 --2- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f54b4077f70 0x7f54b407a430 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f54b8033000 tx=0x7f54b8000fe0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:28.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.703+0000 7f54c659c640 1 --2- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:6818/3616946293,v1:172.21.15.190:6819/3616946293] conn(0x7f54b407e4b0 0x7f54b40808f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:28.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.703+0000 7f54af7fe640 1 -- 172.21.15.98:0/1840381408 <== osd.7 v2:172.21.15.190:6818/3616946293 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f54c0018e50 con 0x7f54b407e4b0 2024-09-10T21:00:28.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.716+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1840381408 --> [v2:172.21.15.190:6818/3616946293,v1:172.21.15.190:6819/3616946293] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f54c80b3e80 con 0x7f54b407e4b0 2024-09-10T21:00:28.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.716+0000 7f54af7fe640 1 -- 172.21.15.98:0/1840381408 <== osd.7 v2:172.21.15.190:6818/3616946293 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f54c80b3e80 con 0x7f54b407e4b0 2024-09-10T21:00:28.720 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-10T21:00:28.720 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:00:28.721 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-10T21:00:28.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.716+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:6818/3616946293,v1:172.21.15.190:6819/3616946293] conn(0x7f54b407e4b0 msgr2=0x7f54b40808f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.716+0000 7f54cd1f4640 1 --2- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:6818/3616946293,v1:172.21.15.190:6819/3616946293] conn(0x7f54b407e4b0 0x7f54b40808f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.717+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f54b4077f70 msgr2=0x7f54b407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.717+0000 7f54cd1f4640 1 --2- 172.21.15.98:0/1840381408 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f54b4077f70 0x7f54b407a430 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f54b8033000 tx=0x7f54b8000fe0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.717+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1840381408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54c813f430 msgr2=0x7f54c813f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.717+0000 7f54cd1f4640 1 --2- 172.21.15.98:0/1840381408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54c813f430 0x7f54c813f8a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f54c0005fd0 tx=0x7f54c0010f40 comp rx=0 tx=0).stop 2024-09-10T21:00:28.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.717+0000 7f54c759e640 1 -- 172.21.15.98:0/1840381408 reap_dead start 2024-09-10T21:00:28.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.717+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1840381408 shutdown_connections 2024-09-10T21:00:28.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.717+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1840381408 >> 172.21.15.98:0/1840381408 conn(0x7f54c80b1760 msgr2=0x7f54c80b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:28.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.717+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1840381408 shutdown_connections 2024-09-10T21:00:28.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.717+0000 7f54cd1f4640 1 -- 172.21.15.98:0/1840381408 wait complete. 2024-09-10T21:00:28.731 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-10T21:00:28.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.828+0000 7fec6edec640 1 -- 172.21.15.98:0/3532619857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815c070 msgr2=0x7fec68074870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.828+0000 7fec6edec640 1 --2- 172.21.15.98:0/3532619857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815c070 0x7fec68074870 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fec60005fd0 tx=0x7fec6002a540 comp rx=0 tx=0).stop 2024-09-10T21:00:28.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.828+0000 7fec6edec640 1 -- 172.21.15.98:0/3532619857 shutdown_connections 2024-09-10T21:00:28.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.828+0000 7fec6edec640 1 --2- 172.21.15.98:0/3532619857 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815c070 0x7fec68074870 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.828+0000 7fec6edec640 1 --2- 172.21.15.98:0/3532619857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec68156330 0x7fec68074330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.828+0000 7fec6edec640 1 -- 172.21.15.98:0/3532619857 >> 172.21.15.98:0/3532619857 conn(0x7fec680b1c80 msgr2=0x7fec680b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:28.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.828+0000 7fec6edec640 1 -- 172.21.15.98:0/3532619857 shutdown_connections 2024-09-10T21:00:28.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.828+0000 7fec6edec640 1 -- 172.21.15.98:0/3532619857 wait complete. 2024-09-10T21:00:28.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.829+0000 7fec6edec640 1 Processor -- start 2024-09-10T21:00:28.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.829+0000 7fec6edec640 1 -- start start 2024-09-10T21:00:28.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.830+0000 7fec6edec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec68155f50 0x7fec680ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.830+0000 7fec6edec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec68156330 0x7fec680d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.830+0000 7fec6edec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815c070 0x7fec6815b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.830+0000 7fec6edec640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fec680707a0 con 0x7fec6815c070 2024-09-10T21:00:28.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.830+0000 7fec6edec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fec68070620 con 0x7fec68155f50 2024-09-10T21:00:28.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.830+0000 7fec6edec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fec68070920 con 0x7fec68156330 2024-09-10T21:00:28.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.830+0000 7fec6d389640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815c070 0x7fec6815b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.830+0000 7fec6d389640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815c070 0x7fec6815b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42188/0 (socket says 172.21.15.98:42188) 2024-09-10T21:00:28.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.830+0000 7fec6d389640 1 -- 172.21.15.98:0/2092167130 learned_addr learned my addr 172.21.15.98:0/2092167130 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:28.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.830+0000 7fec6d389640 1 -- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec68156330 msgr2=0x7fec680d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.830+0000 7fec67fff640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec68156330 0x7fec680d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.830+0000 7fec6cb88640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec68155f50 0x7fec680ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.831+0000 7fec6d389640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec68156330 0x7fec680d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.831+0000 7fec6d389640 1 -- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec68155f50 msgr2=0x7fec680ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:28.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.831+0000 7fec6d389640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec68155f50 0x7fec680ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:28.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.831+0000 7fec6d389640 1 -- 172.21.15.98:0/2092167130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec681f8020 con 0x7fec6815c070 2024-09-10T21:00:28.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.831+0000 7fec67fff640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec68156330 0x7fec680d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:28.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.831+0000 7fec6cb88640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec68155f50 0x7fec680ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:28.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.831+0000 7fec6d389640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815c070 0x7fec6815b430 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fec60006010 tx=0x7fec60003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:28.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.831+0000 7fec657fa640 1 -- 172.21.15.98:0/2092167130 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6003e030 con 0x7fec6815c070 2024-09-10T21:00:28.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.831+0000 7fec657fa640 1 -- 172.21.15.98:0/2092167130 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec60003e50 con 0x7fec6815c070 2024-09-10T21:00:28.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.831+0000 7fec657fa640 1 -- 172.21.15.98:0/2092167130 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60035040 con 0x7fec6815c070 2024-09-10T21:00:28.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.831+0000 7fec6edec640 1 -- 172.21.15.98:0/2092167130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec681f82c0 con 0x7fec6815c070 2024-09-10T21:00:28.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.831+0000 7fec6edec640 1 -- 172.21.15.98:0/2092167130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec681f8880 con 0x7fec6815c070 2024-09-10T21:00:28.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.832+0000 7fec6edec640 1 -- 172.21.15.98:0/2092167130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec68155500 con 0x7fec6815c070 2024-09-10T21:00:28.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.833+0000 7fec657fa640 1 -- 172.21.15.98:0/2092167130 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fec600049e0 con 0x7fec6815c070 2024-09-10T21:00:28.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.833+0000 7fec657fa640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fec48077f70 0x7fec4807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:28.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.833+0000 7fec657fa640 1 -- 172.21.15.98:0/2092167130 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7fec600ba600 con 0x7fec6815c070 2024-09-10T21:00:28.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.833+0000 7fec6cb88640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fec48077f70 0x7fec4807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:28.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.834+0000 7fec6cb88640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fec48077f70 0x7fec4807a430 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fec680ccdd0 tx=0x7fec58009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:28.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:28.837+0000 7fec657fa640 1 -- 172.21.15.98:0/2092167130 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fec68155500 con 0x7fec6815c070 2024-09-10T21:00:29.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.012+0000 7fec6edec640 1 -- 172.21.15.98:0/2092167130 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fec6814f000 con 0x7fec48077f70 2024-09-10T21:00:29.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.013+0000 7fec657fa640 1 -- 172.21.15.98:0/2092167130 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fec6814f000 con 0x7fec48077f70 2024-09-10T21:00:29.185 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 7 to deep-scrub 2024-09-10T21:00:29.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.015+0000 7fec6edec640 1 -- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fec48077f70 msgr2=0x7fec4807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:29.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.015+0000 7fec6edec640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fec48077f70 0x7fec4807a430 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fec680ccdd0 tx=0x7fec58009040 comp rx=0 tx=0).stop 2024-09-10T21:00:29.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.015+0000 7fec6edec640 1 -- 172.21.15.98:0/2092167130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815c070 msgr2=0x7fec6815b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:29.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.015+0000 7fec6edec640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815c070 0x7fec6815b430 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fec60006010 tx=0x7fec60003960 comp rx=0 tx=0).stop 2024-09-10T21:00:29.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.015+0000 7fec6edec640 1 -- 172.21.15.98:0/2092167130 shutdown_connections 2024-09-10T21:00:29.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.015+0000 7fec6edec640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7fec48077f70 0x7fec4807a430 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:29.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.015+0000 7fec6edec640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815c070 0x7fec6815b430 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:29.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.015+0000 7fec6edec640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec68156330 0x7fec680d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:29.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.015+0000 7fec6edec640 1 --2- 172.21.15.98:0/2092167130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec68155f50 0x7fec680ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:29.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.015+0000 7fec6edec640 1 -- 172.21.15.98:0/2092167130 >> 172.21.15.98:0/2092167130 conn(0x7fec680b1c80 msgr2=0x7fec680b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:29.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.015+0000 7fec6edec640 1 -- 172.21.15.98:0/2092167130 shutdown_connections 2024-09-10T21:00:29.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.015+0000 7fec6edec640 1 -- 172.21.15.98:0/2092167130 wait complete. 2024-09-10T21:00:29.186 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:00:29.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.284+0000 7f868556b640 1 -- 172.21.15.98:0/3627520856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86800b4250 msgr2=0x7f86800b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:29.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.284+0000 7f868556b640 1 --2- 172.21.15.98:0/3627520856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86800b4250 0x7f86800b4700 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8678005fd0 tx=0x7f867802a520 comp rx=0 tx=0).stop 2024-09-10T21:00:29.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.284+0000 7f868556b640 1 -- 172.21.15.98:0/3627520856 shutdown_connections 2024-09-10T21:00:29.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.284+0000 7f868556b640 1 --2- 172.21.15.98:0/3627520856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86800cec40 0x7f86800cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:29.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.284+0000 7f868556b640 1 --2- 172.21.15.98:0/3627520856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86800b4250 0x7f86800b4700 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:29.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.284+0000 7f868556b640 1 -- 172.21.15.98:0/3627520856 >> 172.21.15.98:0/3627520856 conn(0x7f868009d190 msgr2=0x7f86800d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:29.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.284+0000 7f868556b640 1 -- 172.21.15.98:0/3627520856 shutdown_connections 2024-09-10T21:00:29.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.284+0000 7f868556b640 1 -- 172.21.15.98:0/3627520856 wait complete. 2024-09-10T21:00:29.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.285+0000 7f868556b640 1 Processor -- start 2024-09-10T21:00:29.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.285+0000 7f868556b640 1 -- start start 2024-09-10T21:00:29.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f868556b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86800b4250 0x7f86800b9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:29.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f868556b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86800cec40 0x7f868014dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:29.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f868556b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86800d08b0 0x7f86800cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:29.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f868556b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f86800b4fd0 con 0x7f86800d08b0 2024-09-10T21:00:29.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f868556b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f86800b4e50 con 0x7f86800b4250 2024-09-10T21:00:29.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f868556b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f86800b5150 con 0x7f86800cec40 2024-09-10T21:00:29.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86800d08b0 0x7f86800cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:29.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86800d08b0 0x7f86800cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42208/0 (socket says 172.21.15.98:42208) 2024-09-10T21:00:29.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f867f7fe640 1 -- 172.21.15.98:0/310658089 learned_addr learned my addr 172.21.15.98:0/310658089 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:29.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f867e7fc640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86800cec40 0x7f868014dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:29.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f867effd640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86800b4250 0x7f86800b9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:29.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f867f7fe640 1 -- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86800cec40 msgr2=0x7f868014dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:29.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f867f7fe640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86800cec40 0x7f868014dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:29.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f867f7fe640 1 -- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86800b4250 msgr2=0x7f86800b9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:29.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f867f7fe640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86800b4250 0x7f86800b9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:29.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f867f7fe640 1 -- 172.21.15.98:0/310658089 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86801f81f0 con 0x7f86800d08b0 2024-09-10T21:00:29.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f867effd640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86800b4250 0x7f86800b9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:00:29.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f867f7fe640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86800d08b0 0x7f86800cd140 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8678009230 tx=0x7f8678003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:29.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.286+0000 7f86677fe640 1 -- 172.21.15.98:0/310658089 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867803f070 con 0x7f86800d08b0 2024-09-10T21:00:29.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.287+0000 7f86677fe640 1 -- 172.21.15.98:0/310658089 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8678004890 con 0x7f86800d08b0 2024-09-10T21:00:29.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.287+0000 7f868556b640 1 -- 172.21.15.98:0/310658089 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86801f8460 con 0x7f86800d08b0 2024-09-10T21:00:29.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.287+0000 7f86677fe640 1 -- 172.21.15.98:0/310658089 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678035030 con 0x7f86800d08b0 2024-09-10T21:00:29.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.287+0000 7f868556b640 1 -- 172.21.15.98:0/310658089 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8680151450 con 0x7f86800d08b0 2024-09-10T21:00:29.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.288+0000 7f868556b640 1 -- 172.21.15.98:0/310658089 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86800b4700 con 0x7f86800d08b0 2024-09-10T21:00:29.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.288+0000 7f86677fe640 1 -- 172.21.15.98:0/310658089 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8678004ce0 con 0x7f86800d08b0 2024-09-10T21:00:29.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.288+0000 7f86677fe640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f866c077f70 0x7f866c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:29.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.289+0000 7f86677fe640 1 -- 172.21.15.98:0/310658089 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f86780bad80 con 0x7f86800d08b0 2024-09-10T21:00:29.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.289+0000 7f867effd640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f866c077f70 0x7f866c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:29.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.289+0000 7f867effd640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f866c077f70 0x7f866c07a430 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8670007e90 tx=0x7f86700073a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:29.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.293+0000 7f86677fe640 1 -- 172.21.15.98:0/310658089 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f86780839a0 con 0x7f86800d08b0 2024-09-10T21:00:29.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.464+0000 7f868556b640 1 -- 172.21.15.98:0/310658089 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f86801f8c60 con 0x7f866c077f70 2024-09-10T21:00:29.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.464+0000 7f86677fe640 1 -- 172.21.15.98:0/310658089 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13454 (secure 0 0 0) 0x7f867000cdb0 con 0x7f866c077f70 2024-09-10T21:00:29.468 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:29.469 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-10T21:00:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.467+0000 7f868556b640 1 -- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f866c077f70 msgr2=0x7f866c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.467+0000 7f868556b640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f866c077f70 0x7f866c07a430 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8670007e90 tx=0x7f86700073a0 comp rx=0 tx=0).stop 2024-09-10T21:00:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.467+0000 7f868556b640 1 -- 172.21.15.98:0/310658089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86800d08b0 msgr2=0x7f86800cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.467+0000 7f868556b640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86800d08b0 0x7f86800cd140 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8678009230 tx=0x7f8678003960 comp rx=0 tx=0).stop 2024-09-10T21:00:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.467+0000 7f868556b640 1 -- 172.21.15.98:0/310658089 shutdown_connections 2024-09-10T21:00:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.467+0000 7f868556b640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f866c077f70 0x7f866c07a430 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.467+0000 7f868556b640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86800d08b0 0x7f86800cd140 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:29.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.467+0000 7f868556b640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86800cec40 0x7f868014dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:29.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.467+0000 7f868556b640 1 --2- 172.21.15.98:0/310658089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86800b4250 0x7f86800b9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:29.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.467+0000 7f868556b640 1 -- 172.21.15.98:0/310658089 >> 172.21.15.98:0/310658089 conn(0x7f868009d190 msgr2=0x7f86800b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:29.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.467+0000 7f868556b640 1 -- 172.21.15.98:0/310658089 shutdown_connections 2024-09-10T21:00:29.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:29.467+0000 7f868556b640 1 -- 172.21.15.98:0/310658089 wait complete. 2024-09-10T21:00:29.482 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-09-10T21:00:28.931740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":221404,"kb_used_data":5160,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753316,"statfs":{"total":773094113280,"available":772867395584,"internally_reserved":0,"allocated":5283840,"data_stored":2969600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.494047"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":154,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:00:21.444837+0000","last_change":"2024-09-10T20:58:36.895993+0000","last_active":"2024-09-10T21:00:21.444837+0000","last_peered":"2024-09-10T21:00:21.444837+0000","last_clean":"2024-09-10T21:00:21.444837+0000","last_became_active":"2024-09-10T20:58:36.894988+0000","last_became_peered":"2024-09-10T20:58:36.894988+0000","last_unstale":"2024-09-10T21:00:21.444837+0000","last_undegraded":"2024-09-10T21:00:21.444837+0000","last_fullsized":"2024-09-10T21:00:21.444837+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:58:35.588702+0000","last_clean_scrub_stamp":"2024-09-10T20:58:35.588702+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:39.082528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1105920,"data_stored":813976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":223608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":223608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":223608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542169,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":223608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542169,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":223608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542169,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1105920,"data_stored":813976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":223608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:00:29.486 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-10T20:58:35.588702+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=20, tm_min=58, tm_sec=35, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=0, tm_sec=24, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:00:29.486 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-10T21:00:44.525 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:00:44.528 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:00:49.486 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:00:49.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.596+0000 7efc5f640640 1 -- 172.21.15.98:0/2288273968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc44004d40 msgr2=0x7efc440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:49.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.596+0000 7efc5f640640 1 --2- 172.21.15.98:0/2288273968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc44004d40 0x7efc440051c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efc50009f00 tx=0x7efc44005830 comp rx=0 tx=0).stop 2024-09-10T21:00:49.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.596+0000 7efc5dbdd640 1 -- 172.21.15.98:0/2288273968 reap_dead start 2024-09-10T21:00:49.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.596+0000 7efc5f640640 1 -- 172.21.15.98:0/2288273968 shutdown_connections 2024-09-10T21:00:49.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.596+0000 7efc5f640640 1 -- 172.21.15.98:0/2288273968 >> 172.21.15.98:0/2288273968 conn(0x7efc580d3f70 msgr2=0x7efc580d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:49.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.596+0000 7efc5f640640 1 -- 172.21.15.98:0/2288273968 shutdown_connections 2024-09-10T21:00:49.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.596+0000 7efc5f640640 1 -- 172.21.15.98:0/2288273968 wait complete. 2024-09-10T21:00:49.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.598+0000 7efc5f640640 1 Processor -- start 2024-09-10T21:00:49.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.598+0000 7efc5f640640 1 -- start start 2024-09-10T21:00:49.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.598+0000 7efc5f640640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc44004d40 0x7efc580ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:49.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.598+0000 7efc5f640640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5815a0f0 0x7efc581576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:49.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.598+0000 7efc5f640640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc5815a570 0x7efc58152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:49.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.598+0000 7efc5f640640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efc580d2660 con 0x7efc5815a570 2024-09-10T21:00:49.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.598+0000 7efc5f640640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7efc580d24e0 con 0x7efc5815a0f0 2024-09-10T21:00:49.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.598+0000 7efc5f640640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7efc580d27e0 con 0x7efc44004d40 2024-09-10T21:00:49.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.598+0000 7efc5dbdd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc5815a570 0x7efc58152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:49.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.598+0000 7efc5dbdd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc5815a570 0x7efc58152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33646/0 (socket says 172.21.15.98:33646) 2024-09-10T21:00:49.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.599+0000 7efc5d3dc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc44004d40 0x7efc580ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:49.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.599+0000 7efc5dbdd640 1 -- 172.21.15.98:0/239239591 learned_addr learned my addr 172.21.15.98:0/239239591 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:49.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.599+0000 7efc5cbdb640 1 --2- 172.21.15.98:0/239239591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5815a0f0 0x7efc581576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:49.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.599+0000 7efc5dbdd640 1 -- 172.21.15.98:0/239239591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc44004d40 msgr2=0x7efc580ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:49.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.599+0000 7efc5dbdd640 1 --2- 172.21.15.98:0/239239591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc44004d40 0x7efc580ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:49.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.599+0000 7efc5dbdd640 1 -- 172.21.15.98:0/239239591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5815a0f0 msgr2=0x7efc581576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:49.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.599+0000 7efc5dbdd640 1 --2- 172.21.15.98:0/239239591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5815a0f0 0x7efc581576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:49.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.599+0000 7efc5dbdd640 1 -- 172.21.15.98:0/239239591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc580cbe20 con 0x7efc5815a570 2024-09-10T21:00:49.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.599+0000 7efc5dbdd640 1 --2- 172.21.15.98:0/239239591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc5815a570 0x7efc58152ed0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efc50036040 tx=0x7efc50033d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:49.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.599+0000 7efc4dffb640 1 -- 172.21.15.98:0/239239591 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50043070 con 0x7efc5815a570 2024-09-10T21:00:49.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.599+0000 7efc4dffb640 1 -- 172.21.15.98:0/239239591 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc50003c60 con 0x7efc5815a570 2024-09-10T21:00:49.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.599+0000 7efc5f640640 1 -- 172.21.15.98:0/239239591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc580cc100 con 0x7efc5815a570 2024-09-10T21:00:49.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.599+0000 7efc4dffb640 1 -- 172.21.15.98:0/239239591 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50038040 con 0x7efc5815a570 2024-09-10T21:00:49.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.599+0000 7efc5f640640 1 -- 172.21.15.98:0/239239591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc580cc6c0 con 0x7efc5815a570 2024-09-10T21:00:49.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.600+0000 7efc5f640640 1 -- 172.21.15.98:0/239239591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc580cea40 con 0x7efc5815a570 2024-09-10T21:00:49.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.601+0000 7efc4dffb640 1 -- 172.21.15.98:0/239239591 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7efc5003b050 con 0x7efc5815a570 2024-09-10T21:00:49.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.601+0000 7efc4dffb640 1 --2- 172.21.15.98:0/239239591 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7efc3c077f70 0x7efc3c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:49.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.601+0000 7efc4dffb640 1 -- 172.21.15.98:0/239239591 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7efc500b9900 con 0x7efc5815a570 2024-09-10T21:00:49.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.601+0000 7efc5d3dc640 1 --2- 172.21.15.98:0/239239591 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7efc3c077f70 0x7efc3c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:49.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.602+0000 7efc5d3dc640 1 --2- 172.21.15.98:0/239239591 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7efc3c077f70 0x7efc3c07a430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efc54000ff0 tx=0x7efc540067a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:49.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.606+0000 7efc4dffb640 1 -- 172.21.15.98:0/239239591 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7efc50081ed0 con 0x7efc5815a570 2024-09-10T21:00:49.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.777+0000 7efc5f640640 1 -- 172.21.15.98:0/239239591 --> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc5808dfa0 con 0x7efc3c077f70 2024-09-10T21:00:49.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.778+0000 7efc4dffb640 1 -- 172.21.15.98:0/239239591 <== mgr.4112 v2:172.21.15.190:6800/3586222139 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13344 (secure 0 0 0) 0x7efc5808dfa0 con 0x7efc3c077f70 2024-09-10T21:00:49.782 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:49.782 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-10T21:00:49.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.780+0000 7efc5f640640 1 -- 172.21.15.98:0/239239591 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7efc3c077f70 msgr2=0x7efc3c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:49.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.780+0000 7efc5f640640 1 --2- 172.21.15.98:0/239239591 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7efc3c077f70 0x7efc3c07a430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efc54000ff0 tx=0x7efc540067a0 comp rx=0 tx=0).stop 2024-09-10T21:00:49.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.780+0000 7efc5f640640 1 -- 172.21.15.98:0/239239591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc5815a570 msgr2=0x7efc58152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:49.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.780+0000 7efc5f640640 1 --2- 172.21.15.98:0/239239591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc5815a570 0x7efc58152ed0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7efc50036040 tx=0x7efc50033d70 comp rx=0 tx=0).stop 2024-09-10T21:00:49.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.780+0000 7efc5f640640 1 -- 172.21.15.98:0/239239591 shutdown_connections 2024-09-10T21:00:49.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.780+0000 7efc5f640640 1 --2- 172.21.15.98:0/239239591 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7efc3c077f70 0x7efc3c07a430 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:49.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.780+0000 7efc5f640640 1 --2- 172.21.15.98:0/239239591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc5815a570 0x7efc58152ed0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:49.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.780+0000 7efc5f640640 1 --2- 172.21.15.98:0/239239591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc5815a0f0 0x7efc581576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:49.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.780+0000 7efc5f640640 1 --2- 172.21.15.98:0/239239591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc44004d40 0x7efc580ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:49.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.780+0000 7efc5f640640 1 -- 172.21.15.98:0/239239591 >> 172.21.15.98:0/239239591 conn(0x7efc580d3f70 msgr2=0x7efc580b89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:49.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.780+0000 7efc5f640640 1 -- 172.21.15.98:0/239239591 shutdown_connections 2024-09-10T21:00:49.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.780+0000 7efc5f640640 1 -- 172.21.15.98:0/239239591 wait complete. 2024-09-10T21:00:49.795 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-10T21:00:48.936071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":221388,"kb_used_data":5160,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753332,"statfs":{"total":773094113280,"available":772867411968,"internally_reserved":0,"allocated":5283840,"data_stored":2969600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002439"},"pg_stats":[{"pgid":"1.0","version":"44'74","reported_seq":165,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:00:29.525423+0000","last_change":"2024-09-10T21:00:29.525365+0000","last_active":"2024-09-10T21:00:29.525423+0000","last_peered":"2024-09-10T21:00:29.525423+0000","last_clean":"2024-09-10T21:00:29.525423+0000","last_became_active":"2024-09-10T20:58:36.894988+0000","last_became_peered":"2024-09-10T20:58:36.894988+0000","last_unstale":"2024-09-10T21:00:29.525423+0000","last_undegraded":"2024-09-10T21:00:29.525423+0000","last_fullsized":"2024-09-10T21:00:29.525423+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'74","last_scrub_stamp":"2024-09-10T21:00:29.525291+0000","last_deep_scrub":"44'74","last_deep_scrub_stamp":"2024-09-10T21:00:29.525291+0000","last_clean_scrub_stamp":"2024-09-10T21:00:29.525291+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:24.986219+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":813976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":223608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":223608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":223608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":223608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":223608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":813976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":223608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:00:49.796 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T21:00:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.893+0000 7f2814b3d640 1 -- 172.21.15.98:0/496892498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f281015c470 msgr2=0x7f281015c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.893+0000 7f2814b3d640 1 --2- 172.21.15.98:0/496892498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f281015c470 0x7f281015c8e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f27f8005fd0 tx=0x7f27f802a530 comp rx=0 tx=0).stop 2024-09-10T21:00:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.893+0000 7f2814b3d640 1 -- 172.21.15.98:0/496892498 shutdown_connections 2024-09-10T21:00:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.893+0000 7f2814b3d640 1 --2- 172.21.15.98:0/496892498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f281006f140 0x7f281006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.893+0000 7f2814b3d640 1 --2- 172.21.15.98:0/496892498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f281015c470 0x7f281015c8e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.893+0000 7f2814b3d640 1 -- 172.21.15.98:0/496892498 >> 172.21.15.98:0/496892498 conn(0x7f28100b1760 msgr2=0x7f28100b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.893+0000 7f2814b3d640 1 -- 172.21.15.98:0/496892498 shutdown_connections 2024-09-10T21:00:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.893+0000 7f2814b3d640 1 -- 172.21.15.98:0/496892498 wait complete. 2024-09-10T21:00:49.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.895+0000 7f2814b3d640 1 Processor -- start 2024-09-10T21:00:49.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.895+0000 7f2814b3d640 1 -- start start 2024-09-10T21:00:49.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f2814b3d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f281006f140 0x7f28100b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:49.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f2814b3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2810073780 0x7f28101fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:49.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f2814b3d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f281015c470 0x7f28101fd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:49.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f2814b3d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f281006fc90 con 0x7f281006f140 2024-09-10T21:00:49.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f2814b3d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f281006fb10 con 0x7f2810073780 2024-09-10T21:00:49.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f2814b3d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f281006fe10 con 0x7f281015c470 2024-09-10T21:00:49.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f280e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f281006f140 0x7f28100b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:49.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f280e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f281006f140 0x7f28100b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33652/0 (socket says 172.21.15.98:33652) 2024-09-10T21:00:49.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f280e59c640 1 -- 172.21.15.98:0/3154890555 learned_addr learned my addr 172.21.15.98:0/3154890555 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:49.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f280ed9d640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f281015c470 0x7f28101fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:49.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f280dd9b640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2810073780 0x7f28101fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:49.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f280e59c640 1 -- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f281015c470 msgr2=0x7f28101fd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:49.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f280e59c640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f281015c470 0x7f28101fd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:49.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f280e59c640 1 -- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2810073780 msgr2=0x7f28101fcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:49.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f280e59c640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2810073780 0x7f28101fcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:49.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f280e59c640 1 -- 172.21.15.98:0/3154890555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28101f9040 con 0x7f281006f140 2024-09-10T21:00:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f280dd9b640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2810073780 0x7f28101fcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:00:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f280e59c640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f281006f140 0x7f28100b3790 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f280000c460 tx=0x7f280000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f2806ffd640 1 -- 172.21.15.98:0/3154890555 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800016020 con 0x7f281006f140 2024-09-10T21:00:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f2806ffd640 1 -- 172.21.15.98:0/3154890555 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2800012960 con 0x7f281006f140 2024-09-10T21:00:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f2814b3d640 1 -- 172.21.15.98:0/3154890555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28101f9320 con 0x7f281006f140 2024-09-10T21:00:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f2806ffd640 1 -- 172.21.15.98:0/3154890555 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800012db0 con 0x7f281006f140 2024-09-10T21:00:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.896+0000 7f2814b3d640 1 -- 172.21.15.98:0/3154890555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28101f98e0 con 0x7f281006f140 2024-09-10T21:00:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.897+0000 7f2814b3d640 1 -- 172.21.15.98:0/3154890555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2810069f60 con 0x7f281006f140 2024-09-10T21:00:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.898+0000 7f2806ffd640 1 -- 172.21.15.98:0/3154890555 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f280001a3b0 con 0x7f281006f140 2024-09-10T21:00:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.898+0000 7f2806ffd640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f27f0077fa0 0x7f27f007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.898+0000 7f2806ffd640 1 -- 172.21.15.98:0/3154890555 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f2800099c20 con 0x7f281006f140 2024-09-10T21:00:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.899+0000 7f280dd9b640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f27f0077fa0 0x7f27f007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.899+0000 7f280dd9b640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f27f0077fa0 0x7f27f007a460 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f27f8033000 tx=0x7f27f8004730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:49.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:49.903+0000 7f2806ffd640 1 -- 172.21.15.98:0/3154890555 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2800020030 con 0x7f281006f140 2024-09-10T21:00:50.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.077+0000 7f2814b3d640 1 -- 172.21.15.98:0/3154890555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f28100cd6d0 con 0x7f281006f140 2024-09-10T21:00:50.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.081+0000 7f2806ffd640 1 -- 172.21.15.98:0/3154890555 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v30)=0 v30) ==== 123+0+0 (secure 0 0 0) 0x7f2800062840 con 0x7f281006f140 2024-09-10T21:00:50.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.082+0000 7f2806ffd640 1 -- 172.21.15.98:0/3154890555 <== mon.0 v2:172.21.15.98:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f280009f020 con 0x7f281006f140 2024-09-10T21:00:50.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.084+0000 7f2814b3d640 1 -- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f27f0077fa0 msgr2=0x7f27f007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:50.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.084+0000 7f2814b3d640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f27f0077fa0 0x7f27f007a460 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f27f8033000 tx=0x7f27f8004730 comp rx=0 tx=0).stop 2024-09-10T21:00:50.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.084+0000 7f2814b3d640 1 -- 172.21.15.98:0/3154890555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f281006f140 msgr2=0x7f28100b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:50.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.084+0000 7f2814b3d640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f281006f140 0x7f28100b3790 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f280000c460 tx=0x7f280000c920 comp rx=0 tx=0).stop 2024-09-10T21:00:50.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.084+0000 7f2814b3d640 1 -- 172.21.15.98:0/3154890555 shutdown_connections 2024-09-10T21:00:50.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.084+0000 7f2814b3d640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f27f0077fa0 0x7f27f007a460 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:50.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.084+0000 7f2814b3d640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f281015c470 0x7f28101fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:50.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.084+0000 7f2814b3d640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2810073780 0x7f28101fcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:50.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.084+0000 7f2814b3d640 1 --2- 172.21.15.98:0/3154890555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f281006f140 0x7f28100b3790 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:50.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.084+0000 7f2814b3d640 1 -- 172.21.15.98:0/3154890555 >> 172.21.15.98:0/3154890555 conn(0x7f28100b1760 msgr2=0x7f28100b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:50.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.084+0000 7f2814b3d640 1 -- 172.21.15.98:0/3154890555 shutdown_connections 2024-09-10T21:00:50.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.084+0000 7f2814b3d640 1 -- 172.21.15.98:0/3154890555 wait complete. 2024-09-10T21:00:50.099 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:00:50.099 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:00:50.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.196+0000 7f8748784640 1 -- 172.21.15.98:0/2205745709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8740073780 msgr2=0x7f8740073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:50.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.196+0000 7f8748784640 1 --2- 172.21.15.98:0/2205745709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8740073780 0x7f8740073b60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f87400cfaa0 tx=0x7f873002aa30 comp rx=0 tx=0).stop 2024-09-10T21:00:50.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.196+0000 7f8748784640 1 -- 172.21.15.98:0/2205745709 shutdown_connections 2024-09-10T21:00:50.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.196+0000 7f8748784640 1 --2- 172.21.15.98:0/2205745709 >> v2:172.21.15.190:3300/0 conn(0x7f8740072560 0x7f87400729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:50.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.196+0000 7f8748784640 1 --2- 172.21.15.98:0/2205745709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f874015c470 0x7f874015c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:50.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.196+0000 7f8748784640 1 --2- 172.21.15.98:0/2205745709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8740073780 0x7f8740073b60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:50.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.196+0000 7f8748784640 1 -- 172.21.15.98:0/2205745709 >> 172.21.15.98:0/2205745709 conn(0x7f87400b1760 msgr2=0x7f87400b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:50.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.196+0000 7f8748784640 1 -- 172.21.15.98:0/2205745709 shutdown_connections 2024-09-10T21:00:50.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.196+0000 7f8748784640 1 -- 172.21.15.98:0/2205745709 wait complete. 2024-09-10T21:00:50.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.197+0000 7f8748784640 1 Processor -- start 2024-09-10T21:00:50.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.197+0000 7f8748784640 1 -- start start 2024-09-10T21:00:50.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8748784640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8740072560 0x7f87401e55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:50.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8748784640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8740073780 0x7f87401e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:50.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8748784640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f874015c470 0x7f87401f6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:50.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8748784640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f874015cf70 con 0x7f874015c470 2024-09-10T21:00:50.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8748784640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f874015cdf0 con 0x7f8740073780 2024-09-10T21:00:50.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8748784640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8740072f30 con 0x7f8740072560 2024-09-10T21:00:50.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8746d21640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f874015c470 0x7f87401f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:50.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8746d21640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f874015c470 0x7f87401f6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47186/0 (socket says 172.21.15.98:47186) 2024-09-10T21:00:50.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8746d21640 1 -- 172.21.15.98:0/1852078030 learned_addr learned my addr 172.21.15.98:0/1852078030 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-10T21:00:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8746d21640 1 -- 172.21.15.98:0/1852078030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8740072560 msgr2=0x7f87401e55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8746d21640 1 --2- 172.21.15.98:0/1852078030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8740072560 0x7f87401e55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8746d21640 1 -- 172.21.15.98:0/1852078030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8740073780 msgr2=0x7f87401e5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8746d21640 1 --2- 172.21.15.98:0/1852078030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8740073780 0x7f87401e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8746d21640 1 -- 172.21.15.98:0/1852078030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874012ee90 con 0x7f874015c470 2024-09-10T21:00:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f8746d21640 1 --2- 172.21.15.98:0/1852078030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f874015c470 0x7f87401f6270 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8738009fd0 tx=0x7f873800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.198+0000 7f873effd640 1 -- 172.21.15.98:0/1852078030 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8738011830 con 0x7f874015c470 2024-09-10T21:00:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.199+0000 7f873effd640 1 -- 172.21.15.98:0/1852078030 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f8738016070 con 0x7f874015c470 2024-09-10T21:00:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.199+0000 7f8748784640 1 -- 172.21.15.98:0/1852078030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874012e8d0 con 0x7f874015c470 2024-09-10T21:00:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.199+0000 7f873effd640 1 -- 172.21.15.98:0/1852078030 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8738011c70 con 0x7f874015c470 2024-09-10T21:00:50.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.199+0000 7f8748784640 1 -- 172.21.15.98:0/1852078030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8740154680 con 0x7f874015c470 2024-09-10T21:00:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.199+0000 7f8748784640 1 -- 172.21.15.98:0/1852078030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8740073b60 con 0x7f874015c470 2024-09-10T21:00:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.200+0000 7f873effd640 1 -- 172.21.15.98:0/1852078030 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f873801b460 con 0x7f874015c470 2024-09-10T21:00:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.201+0000 7f873effd640 1 --2- 172.21.15.98:0/1852078030 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f872c077f70 0x7f872c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.201+0000 7f873effd640 1 -- 172.21.15.98:0/1852078030 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5421+0+0 (secure 0 0 0) 0x7f87380998e0 con 0x7f874015c470 2024-09-10T21:00:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.201+0000 7f8746520640 1 --2- 172.21.15.98:0/1852078030 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f872c077f70 0x7f872c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.202+0000 7f8746520640 1 --2- 172.21.15.98:0/1852078030 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f872c077f70 0x7f872c07a430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8730035000 tx=0x7f873002af30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:50.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.205+0000 7f873effd640 1 -- 172.21.15.98:0/1852078030 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8738062590 con 0x7f874015c470 2024-09-10T21:00:50.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.417+0000 7f8748784640 1 -- 172.21.15.98:0/1852078030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87400cd6d0 con 0x7f874015c470 2024-09-10T21:00:50.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.417+0000 7f873effd640 1 -- 172.21.15.98:0/1852078030 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+3917 (secure 0 0 0) 0x7f8738061e90 con 0x7f874015c470 2024-09-10T21:00:50.422 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-10T21:00:50.423 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":25,"btime":"2024-09-10T21:00:19:125045+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/1700293059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":1700293059},{"type":"v1","addr":"172.21.15.98:6834","nonce":1700293059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4048376378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4048376378},{"type":"v1","addr":"172.21.15.190:6835","nonce":4048376378}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4838,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/1268135915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":1268135915},{"type":"v1","addr":"172.21.15.190:6837","nonce":1268135915}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/2245019178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2245019178},{"type":"v1","addr":"172.21.15.98:6835","nonce":2245019178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-10T21:00:50.423 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 25 2024-09-10T21:00:50.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.420+0000 7f8748784640 1 -- 172.21.15.98:0/1852078030 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f872c077f70 msgr2=0x7f872c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:50.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.420+0000 7f8748784640 1 --2- 172.21.15.98:0/1852078030 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f872c077f70 0x7f872c07a430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8730035000 tx=0x7f873002af30 comp rx=0 tx=0).stop 2024-09-10T21:00:50.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.420+0000 7f8748784640 1 -- 172.21.15.98:0/1852078030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f874015c470 msgr2=0x7f87401f6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:50.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.420+0000 7f8748784640 1 --2- 172.21.15.98:0/1852078030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f874015c470 0x7f87401f6270 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8738009fd0 tx=0x7f873800b8e0 comp rx=0 tx=0).stop 2024-09-10T21:00:50.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.420+0000 7f8748784640 1 -- 172.21.15.98:0/1852078030 shutdown_connections 2024-09-10T21:00:50.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.420+0000 7f8748784640 1 --2- 172.21.15.98:0/1852078030 >> [v2:172.21.15.190:6800/3586222139,v1:172.21.15.190:6801/3586222139] conn(0x7f872c077f70 0x7f872c07a430 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:50.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.420+0000 7f8748784640 1 --2- 172.21.15.98:0/1852078030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f874015c470 0x7f87401f6270 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:50.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.420+0000 7f8748784640 1 --2- 172.21.15.98:0/1852078030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8740073780 0x7f87401e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:50.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.420+0000 7f8748784640 1 --2- 172.21.15.98:0/1852078030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8740072560 0x7f87401e55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:50.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.420+0000 7f8748784640 1 -- 172.21.15.98:0/1852078030 >> 172.21.15.98:0/1852078030 conn(0x7f87400b1760 msgr2=0x7f87400cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:50.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.420+0000 7f8748784640 1 -- 172.21.15.98:0/1852078030 shutdown_connections 2024-09-10T21:00:50.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-10T21:00:50.420+0000 7f8748784640 1 -- 172.21.15.98:0/1852078030 wait complete. 2024-09-10T21:00:50.435 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-10T21:00:50.435 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T21:00:50.436 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:00:50.436 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:50.445 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:00:50.445 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:00:50.445 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:50.494 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:00:50.495 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:00:50.495 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:50.509 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:00:50.509 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:00:50.509 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:50.560 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:00:50.560 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T21:00:50.560 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T21:00:50.561 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:50.588 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T21:00:50.588 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T21:00:50.588 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:50.652 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T21:00:50.652 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T21:00:50.652 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:50.721 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T21:00:50.722 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-10T21:00:50.722 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:50.787 INFO:tasks.ceph.osd.3:Stopped 2024-09-10T21:00:50.788 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-10T21:00:50.788 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:50.817 INFO:tasks.ceph.osd.4:Stopped 2024-09-10T21:00:50.817 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-10T21:00:50.817 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:50.896 INFO:tasks.ceph.osd.5:Stopped 2024-09-10T21:00:50.896 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-10T21:00:50.896 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:50.969 INFO:tasks.ceph.osd.6:Stopped 2024-09-10T21:00:50.969 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-10T21:00:50.969 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:51.037 INFO:tasks.ceph.osd.7:Stopped 2024-09-10T21:00:51.038 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T21:00:51.039 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-10T21:00:51.039 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:51.067 INFO:tasks.ceph.mgr.y:Stopped 2024-09-10T21:00:51.067 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T21:00:51.067 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:51.107 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T21:00:51.107 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T21:00:51.108 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T21:00:51.108 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:51.119 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T21:00:51.119 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T21:00:51.119 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:51.160 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T21:00:51.160 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T21:00:51.160 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:00:51.212 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T21:00:51.212 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T21:00:51.212 DEBUG:teuthology.orchestra.run.smithi098:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'Replacing daemon mds.a' | head -n 1 2024-09-10T21:00:51.248 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T21:00:51.248 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T21:00:51.437 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T21:00:51.438 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T21:00:51.527 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T21:00:51.527 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T21:00:51.589 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T21:00:51.589 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-10T21:00:51.642 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-10T21:00:51.642 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-10T21:00:51.798 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-10T21:00:51.798 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-10T21:00:51.894 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-10T21:00:51.894 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-10T21:00:51.951 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi190.front.sepia.ceph.com 2024-09-10T21:00:51.952 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-10T21:00:52.008 INFO:tasks.ceph:Archiving mon data... 2024-09-10T21:00:52.009 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899652/data/mon.a.tgz 2024-09-10T21:00:52.010 DEBUG:teuthology.orchestra.run.smithi098:> mktemp 2024-09-10T21:00:52.025 INFO:teuthology.orchestra.run.smithi098.stdout:/tmp/tmp.eYjNdDT7JW 2024-09-10T21:00:52.026 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.eYjNdDT7JW 2024-09-10T21:00:52.161 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0666 /tmp/tmp.eYjNdDT7JW 2024-09-10T21:00:52.244 DEBUG:teuthology.orchestra.remote:smithi098:/tmp/tmp.eYjNdDT7JW is 318KB 2024-09-10T21:00:52.341 DEBUG:teuthology.orchestra.run.smithi098:> rm -fr /tmp/tmp.eYjNdDT7JW 2024-09-10T21:00:52.356 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899652/data/mon.b.tgz 2024-09-10T21:00:52.356 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-09-10T21:00:52.375 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.FeEuGMnDsL 2024-09-10T21:00:52.375 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.FeEuGMnDsL 2024-09-10T21:00:52.535 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.FeEuGMnDsL 2024-09-10T21:00:52.623 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.FeEuGMnDsL is 425KB 2024-09-10T21:00:52.720 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.FeEuGMnDsL 2024-09-10T21:00:52.736 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899652/data/mon.c.tgz 2024-09-10T21:00:52.736 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-09-10T21:00:52.791 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.HsaPq1j7FS 2024-09-10T21:00:52.791 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.HsaPq1j7FS 2024-09-10T21:00:52.932 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.HsaPq1j7FS 2024-09-10T21:00:53.016 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.HsaPq1j7FS is 424KB 2024-09-10T21:00:53.112 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.HsaPq1j7FS 2024-09-10T21:00:53.127 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T21:00:53.127 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T21:00:53.130 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T21:00:53.212 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T21:00:53.247 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T21:00:53.247 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T21:00:53.249 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899652/remote/smithi098/crash 2024-09-10T21:00:53.249 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T21:00:53.279 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899652/remote/smithi190/crash 2024-09-10T21:00:53.280 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T21:00:53.311 INFO:tasks.ceph:Shutting down logrotate 2024-09-10T21:00:53.311 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:00:53.324 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:00:53.407 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T21:00:53.433 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T21:00:53.457 INFO:tasks.ceph:Compressing logs... 2024-09-10T21:00:53.457 DEBUG:teuthology.orchestra.run.smithi098:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T21:00:53.475 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T21:00:53.497 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T21:00:53.498 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45533.log 2024-09-10T21:00:53.498 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45166.log 2024-09-10T21:00:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42649.log 2024-09-10T21:00:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-10T21:00:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36280.log 2024-09-10T21:00:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.1.42649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42649.log.gz 2024-09-10T21:00:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43047.log 2024-09-10T21:00:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-client.admin.36280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36360.log 2024-09-10T21:00:53.540 INFO:teuthology.orchestra.run.smithi190.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36280.log.gz 2024-09-10T21:00:53.540 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.3.43047.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-10T21:00:53.541 INFO:teuthology.orchestra.run.smithi190.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.43047.log.gz 2024-09-10T21:00:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36440.log 2024-09-10T21:00:53.541 INFO:teuthology.orchestra.run.smithi190.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36360.log.gz 2024-09-10T21:00:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36400.log 2024-09-10T21:00:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36440.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-10T21:00:53.541 INFO:teuthology.orchestra.run.smithi190.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36440.log.gz 2024-09-10T21:00:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36400.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T21:00:53.541 INFO:teuthology.orchestra.run.smithi190.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36400.log.gz 2024-09-10T21:00:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T21:00:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36320.log 2024-09-10T21:00:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36200.log 2024-09-10T21:00:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36320.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-10T21:00:53.541 INFO:teuthology.orchestra.run.smithi190.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36320.log.gz 2024-09-10T21:00:53.541 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36200.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-10T21:00:53.542 INFO:teuthology.orchestra.run.smithi190.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36200.log.gz 2024-09-10T21:00:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T21:00:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-10T21:00:53.542 INFO:teuthology.orchestra.run.smithi190.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T21:00:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T21:00:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mds.d.log: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T21:00:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36240.log 2024-09-10T21:00:53.542 INFO:teuthology.orchestra.run.smithi190.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-10T21:00:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T21:00:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36240.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36240.log.gz 2024-09-10T21:00:53.542 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36151.log 2024-09-10T21:00:53.552 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mon.b.log: 90.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-10T21:00:53.552 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.36151.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36151.log.gz 2024-09-10T21:00:53.581 INFO:teuthology.orchestra.run.smithi190.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T21:00:53.585 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47523.log 2024-09-10T21:00:53.585 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45533.log.gz 2024-09-10T21:00:53.585 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42911.log 2024-09-10T21:00:53.586 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45166.log.gz 2024-09-10T21:00:53.586 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47775.log 2024-09-10T21:00:53.586 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47523.log: /var/log/ceph/ceph-client.admin.42911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49178.log 2024-09-10T21:00:53.586 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47523.log.gz 2024-09-10T21:00:53.586 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42911.log.gz 2024-09-10T21:00:53.586 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48460.log 2024-09-10T21:00:53.586 INFO:teuthology.orchestra.run.smithi098.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47775.log.gz 2024-09-10T21:00:53.586 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42995.log 2024-09-10T21:00:53.586 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49178.log.gz 2024-09-10T21:00:53.586 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44494.log 2024-09-10T21:00:53.586 INFO:teuthology.orchestra.run.smithi098.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.48460.log.gz 2024-09-10T21:00:53.587 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45842.log 2024-09-10T21:00:53.587 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44494.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42995.log.gz 2024-09-10T21:00:53.587 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47029.log 2024-09-10T21:00:53.587 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44494.log.gz 2024-09-10T21:00:53.587 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36586.log 2024-09-10T21:00:53.587 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45842.log.gz 2024-09-10T21:00:53.587 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36326.log 2024-09-10T21:00:53.587 INFO:teuthology.orchestra.run.smithi098.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.47029.log.gz 2024-09-10T21:00:53.587 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49837.log 2024-09-10T21:00:53.587 INFO:teuthology.orchestra.run.smithi098.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.36586.log.gz 2024-09-10T21:00:53.587 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48999.log 2024-09-10T21:00:53.587 INFO:teuthology.orchestra.run.smithi098.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36326.log.gz 2024-09-10T21:00:53.587 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42496.log 2024-09-10T21:00:53.587 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49837.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49837.log.gz 2024-09-10T21:00:53.588 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44482.log 2024-09-10T21:00:53.588 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48999.log.gz 2024-09-10T21:00:53.588 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49221.log 2024-09-10T21:00:53.588 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42496.log.gz 2024-09-10T21:00:53.588 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44482.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30025.log 2024-09-10T21:00:53.588 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44482.log.gz 2024-09-10T21:00:53.588 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42699.log 2024-09-10T21:00:53.588 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph.tmp-client.admin.30025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43718.log 2024-09-10T21:00:53.588 INFO:teuthology.orchestra.run.smithi098.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30025.log.gz 2024-09-10T21:00:53.588 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42699.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49221.log.gz 2024-09-10T21:00:53.588 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49794.log 2024-09-10T21:00:53.588 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42699.log.gz 2024-09-10T21:00:53.589 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43718.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T21:00:53.589 INFO:teuthology.orchestra.run.smithi098.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43718.log.gz 2024-09-10T21:00:53.589 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50337.log 2024-09-10T21:00:53.589 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3%/var/log/ceph/ceph-osd.1.log: -- replaced with /var/log/ceph/ceph-client.admin.49794.log.gz 2024-09-10T21:00:53.589 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44493.log 2024-09-10T21:00:53.589 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48522.log 2024-09-10T21:00:53.589 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50337.log.gz 2024-09-10T21:00:53.589 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47481.log 2024-09-10T21:00:53.589 INFO:teuthology.orchestra.run.smithi098.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44493.log.gz 2024-09-10T21:00:53.589 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47649.log 2024-09-10T21:00:53.589 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48522.log: /var/log/ceph/ceph-client.admin.47481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48789.log 2024-09-10T21:00:53.589 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48522.log.gz 2024-09-10T21:00:53.589 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47481.log.gz 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48831.log 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47649.log.gz 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51254.log 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48789.log.gz 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45168.log 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48831.log: /var/log/ceph/ceph-client.admin.51254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49287.log 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48831.log.gz 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51254.log.gz 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44315.log 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45168.log.gz 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43853.log 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49287.log: /var/log/ceph/ceph-client.admin.44315.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49287.log.gz 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51338.log 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44315.log.gz 2024-09-10T21:00:53.590 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47565.log 2024-09-10T21:00:53.591 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43853.log: /var/log/ceph/ceph-client.admin.51338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45449.log 2024-09-10T21:00:53.591 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43853.log.gz 2024-09-10T21:00:53.591 INFO:teuthology.orchestra.run.smithi098.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51338.log.gz 2024-09-10T21:00:53.591 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47404.log 2024-09-10T21:00:53.591 INFO:teuthology.orchestra.run.smithi098.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47565.log.gz 2024-09-10T21:00:53.591 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51548.log 2024-09-10T21:00:53.591 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45449.log.gz 2024-09-10T21:00:53.591 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50864.log 2024-09-10T21:00:53.591 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47404.log: /var/log/ceph/ceph-client.admin.51548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49633.log 2024-09-10T21:00:53.591 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47404.log.gz 2024-09-10T21:00:53.591 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51548.log.gz 2024-09-10T21:00:53.591 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51590.log 2024-09-10T21:00:53.591 INFO:teuthology.orchestra.run.smithi098.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50864.log.gz 2024-09-10T21:00:53.591 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49041.log 2024-09-10T21:00:53.592 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49633.log: /var/log/ceph/ceph-client.admin.51590.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49633.log.gz 2024-09-10T21:00:53.592 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44495.log 2024-09-10T21:00:53.592 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51590.log.gz 2024-09-10T21:00:53.592 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43937.log 2024-09-10T21:00:53.592 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49041.log: /var/log/ceph/ceph-client.admin.44495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50253.log 2024-09-10T21:00:53.592 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49041.log.gz 2024-09-10T21:00:53.592 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4%/var/log/ceph/ceph-client.admin.43937.log: -- replaced with /var/log/ceph/ceph-client.admin.44495.log.gz 2024-09-10T21:00:53.592 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49083.log 2024-09-10T21:00:53.592 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43937.log.gz 2024-09-10T21:00:53.592 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48250.log 2024-09-10T21:00:53.592 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50253.log.gz 2024-09-10T21:00:53.592 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44490.log 2024-09-10T21:00:53.592 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36368.log 2024-09-10T21:00:53.592 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49083.log.gz 2024-09-10T21:00:53.593 INFO:teuthology.orchestra.run.smithi098.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48250.log.gz 2024-09-10T21:00:53.593 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46735.log 2024-09-10T21:00:53.593 INFO:teuthology.orchestra.run.smithi098.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44490.log.gz 2024-09-10T21:00:53.593 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50505.log 2024-09-10T21:00:53.593 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36368.log: /var/log/ceph/ceph-client.admin.46735.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36368.log.gz 2024-09-10T21:00:53.593 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50991.log 2024-09-10T21:00:53.593 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46735.log.gz 2024-09-10T21:00:53.593 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46297.log 2024-09-10T21:00:53.593 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50505.log: /var/log/ceph/ceph-client.admin.50991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47691.log 2024-09-10T21:00:53.593 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50505.log.gz 2024-09-10T21:00:53.593 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50991.log.gz 2024-09-10T21:00:53.593 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45088.log 2024-09-10T21:00:53.593 INFO:teuthology.orchestra.run.smithi098.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46297.log.gz 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49571.log 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47691.log.gz 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46903.log 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45088.log: /var/log/ceph/ceph-client.admin.49571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51044.log 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45088.log.gz 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49571.log.gz 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46903.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46903.log.gz 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45491.log 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51044.log: /var/log/ceph/ceph.audit.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51044.log.gz 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47878.log 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45491.log: /var/log/ceph/ceph-client.admin.48915.log 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47878.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45491.log.gz 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51674.log 2024-09-10T21:00:53.594 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47878.log.gz 2024-09-10T21:00:53.595 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48915.log: 89.0%gzip -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T21:00:53.595 INFO:teuthology.orchestra.run.smithi098.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51776.log 2024-09-10T21:00:53.595 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48915.log.gz 2024-09-10T21:00:53.595 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49475.log 2024-09-10T21:00:53.595 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51674.log: /var/log/ceph/ceph-client.admin.51776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44021.log 2024-09-10T21:00:53.595 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51674.log.gz 2024-09-10T21:00:53.595 INFO:teuthology.orchestra.run.smithi098.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.51776.log.gz 2024-09-10T21:00:53.595 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.0.49475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49475.log.gz 2024-09-10T21:00:53.595 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45407.log 2024-09-10T21:00:53.595 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47314.log 2024-09-10T21:00:53.595 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44021.log: /var/log/ceph/ceph-client.admin.45407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37057.log 2024-09-10T21:00:53.596 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44021.log.gz 2024-09-10T21:00:53.596 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45407.log.gz 2024-09-10T21:00:53.596 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49734.log 2024-09-10T21:00:53.596 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47314.log.gz 2024-09-10T21:00:53.596 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44496.log 2024-09-10T21:00:53.596 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50211.log 86.8% 2024-09-10T21:00:53.596 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37057.log.gz 2024-09-10T21:00:53.596 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44496.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49734.log.gz 2024-09-10T21:00:53.596 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44828.log 2024-09-10T21:00:53.596 INFO:teuthology.orchestra.run.smithi098.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44496.log.gz 2024-09-10T21:00:53.596 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51380.log 2024-09-10T21:00:53.596 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50211.log.gz 2024-09-10T21:00:53.597 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43037.log 2024-09-10T21:00:53.597 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44828.log: /var/log/ceph/ceph-client.admin.51380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36609.log 2024-09-10T21:00:53.597 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44828.log.gz 2024-09-10T21:00:53.597 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51380.log.gz 2024-09-10T21:00:53.597 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45085.log 2024-09-10T21:00:53.597 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43979.log 2024-09-10T21:00:53.597 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36609.log: /var/log/ceph/ceph-client.admin.45085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43776.log 2024-09-10T21:00:53.597 INFO:teuthology.orchestra.run.smithi098.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.36609.log.gz 2024-09-10T21:00:53.597 INFO:teuthology.orchestra.run.smithi098.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43037.log.gz 2024-09-10T21:00:53.597 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45085.log.gz 2024-09-10T21:00:53.597 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51716.log 2024-09-10T21:00:53.597 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50631.log 2024-09-10T21:00:53.597 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43979.log.gz 2024-09-10T21:00:53.597 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51716.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43776.log.gz 2024-09-10T21:00:53.598 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50589.log 2024-09-10T21:00:53.598 INFO:teuthology.orchestra.run.smithi098.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51716.log.gz 2024-09-10T21:00:53.598 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36599.log 2024-09-10T21:00:53.598 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50631.log.gz 2024-09-10T21:00:53.598 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42827.log 2024-09-10T21:00:53.598 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50589.log: /var/log/ceph/ceph-client.admin.36599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36856.log 2024-09-10T21:00:53.598 INFO:teuthology.orchestra.run.smithi098.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50589.log.gz 2024-09-10T21:00:53.598 INFO:teuthology.orchestra.run.smithi098.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.36599.log.gz 2024-09-10T21:00:53.598 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36588.log 2024-09-10T21:00:53.598 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48208.log 2024-09-10T21:00:53.598 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42827.log.gz 2024-09-10T21:00:53.598 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36856.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36856.log.gz 2024-09-10T21:00:53.599 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47235.log 2024-09-10T21:00:53.599 INFO:teuthology.orchestra.run.smithi098.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36588.log.gz 2024-09-10T21:00:53.599 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42454.log 2024-09-10T21:00:53.599 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48208.log: /var/log/ceph/ceph-client.admin.47235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42412.log 2024-09-10T21:00:53.599 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48208.log.gz 2024-09-10T21:00:53.599 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47235.log.gz 2024-09-10T21:00:53.599 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51632.log 2024-09-10T21:00:53.599 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42454.log.gz 2024-09-10T21:00:53.599 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-10T21:00:53.599 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42412.log: /var/log/ceph/ceph-client.admin.51632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47733.log 2024-09-10T21:00:53.599 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42412.log.gz 2024-09-10T21:00:53.599 INFO:teuthology.orchestra.run.smithi098.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51632.log.gz 2024-09-10T21:00:53.599 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47962.log 2024-09-10T21:00:53.599 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36194.log 2024-09-10T21:00:53.600 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47733.log.gz 2024-09-10T21:00:53.600 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49981.log 2024-09-10T21:00:53.600 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47962.log.gz 2024-09-10T21:00:53.600 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44833.log 2024-09-10T21:00:53.600 INFO:teuthology.orchestra.run.smithi098.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36194.log.gz 2024-09-10T21:00:53.600 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46945.log 2024-09-10T21:00:53.600 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49981.log: /var/log/ceph/ceph-client.admin.44833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43895.log 2024-09-10T21:00:53.600 INFO:teuthology.orchestra.run.smithi098.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.44833.log.gz 2024-09-10T21:00:53.600 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46945.log: gzip 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49981.log.gz 2024-09-10T21:00:53.600 INFO:teuthology.orchestra.run.smithi098.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44831.log 2024-09-10T21:00:53.600 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46945.log.gz 2024-09-10T21:00:53.600 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42743.log 2024-09-10T21:00:53.601 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43895.log.gz 2024-09-10T21:00:53.601 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48166.log 2024-09-10T21:00:53.601 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44831.log.gz 2024-09-10T21:00:53.601 INFO:teuthology.orchestra.run.smithi098.stderr:gzip/var/log/ceph/ceph-client.admin.42743.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47920.log 2024-09-10T21:00:53.601 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48166.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42743.log.gz 2024-09-10T21:00:53.601 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44273.log 2024-09-10T21:00:53.601 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48166.log.gz 2024-09-10T21:00:53.601 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48566.log 2024-09-10T21:00:53.601 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47920.log.gz 2024-09-10T21:00:53.601 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50547.log 2024-09-10T21:00:53.601 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44273.log.gz 2024-09-10T21:00:53.601 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45617.log 2024-09-10T21:00:53.601 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48566.log.gz 2024-09-10T21:00:53.602 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48082.log 2024-09-10T21:00:53.602 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50547.log.gz 2024-09-10T21:00:53.602 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42234.log 2024-09-10T21:00:53.602 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45617.log.gz 2024-09-10T21:00:53.602 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42602.log 2024-09-10T21:00:53.602 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48082.log.gz 2024-09-10T21:00:53.602 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42785.log 2024-09-10T21:00:53.602 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42234.log.gz 2024-09-10T21:00:53.602 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48747.log 2024-09-10T21:00:53.602 INFO:teuthology.orchestra.run.smithi098.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42602.log.gz 2024-09-10T21:00:53.602 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46777.log 2024-09-10T21:00:53.602 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42785.log.gz 2024-09-10T21:00:53.602 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36222.log 2024-09-10T21:00:53.603 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48747.log.gz 2024-09-10T21:00:53.603 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48124.log 2024-09-10T21:00:53.603 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46777.log.gz 2024-09-10T21:00:53.603 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50421.log 2024-09-10T21:00:53.603 INFO:teuthology.orchestra.run.smithi098.stderr: 77.1% -- replaced with /var/log/ceph/ceph-client.admin.36222.log.gz 2024-09-10T21:00:53.603 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36578.log 2024-09-10T21:00:53.603 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48124.log.gz 2024-09-10T21:00:53.603 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50421.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T21:00:53.603 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50421.log.gz 2024-09-10T21:00:53.603 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36610.log 2024-09-10T21:00:53.603 INFO:teuthology.orchestra.run.smithi098.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.36578.log.gz 2024-09-10T21:00:53.603 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51464.log 2024-09-10T21:00:53.604 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36471.log 2024-09-10T21:00:53.604 INFO:teuthology.orchestra.run.smithi098.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36610.log.gz/var/log/ceph/ceph-client.admin.51464.log: gzip 2024-09-10T21:00:53.604 INFO:teuthology.orchestra.run.smithi098.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47122.log 2024-09-10T21:00:53.604 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51464.log.gz 2024-09-10T21:00:53.604 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49921.log 2024-09-10T21:00:53.604 INFO:teuthology.orchestra.run.smithi098.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.36471.log.gz 2024-09-10T21:00:53.604 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45169.log 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47122.log.gz 2024-09-10T21:00:53.604 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-10T21:00:53.604 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49921.log: gzip -5 --verbose -- 87.1% /var/log/ceph/ceph-client.admin.51506.log 2024-09-10T21:00:53.604 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49921.log.gz 2024-09-10T21:00:53.604 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50673.log 2024-09-10T21:00:53.604 INFO:teuthology.orchestra.run.smithi098.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45169.log.gz 2024-09-10T21:00:53.604 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45167.log 2024-09-10T21:00:53.604 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51506.log.gz 2024-09-10T21:00:53.605 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46819.log 2024-09-10T21:00:53.605 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50673.log.gz 2024-09-10T21:00:53.605 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51128.log 2024-09-10T21:00:53.605 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45167.log.gz 2024-09-10T21:00:53.605 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46819.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T21:00:53.605 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46819.log.gz 2024-09-10T21:00:53.605 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42558.log 2024-09-10T21:00:53.605 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51128.log.gz 2024-09-10T21:00:53.605 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44063.log 2024-09-10T21:00:53.605 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36582.log 2024-09-10T21:00:53.605 INFO:teuthology.orchestra.run.smithi098.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42558.log.gz/var/log/ceph/ceph-client.admin.44063.log: 2024-09-10T21:00:53.605 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36250.log 2024-09-10T21:00:53.605 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44063.log.gz 2024-09-10T21:00:53.606 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51170.log 2024-09-10T21:00:53.606 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36582.log: /var/log/ceph/ceph-client.admin.36250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51816.log 2024-09-10T21:00:53.606 INFO:teuthology.orchestra.run.smithi098.stderr: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.36250.log.gz 2024-09-10T21:00:53.606 INFO:teuthology.orchestra.run.smithi098.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36582.log.gz 2024-09-10T21:00:53.606 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37148.log 2024-09-10T21:00:53.606 INFO:teuthology.orchestra.run.smithi098.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51170.log.gz 2024-09-10T21:00:53.606 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51816.log: -- /var/log/ceph/ceph-client.admin.51296.log 2024-09-10T21:00:53.606 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37148.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51816.log.gz 2024-09-10T21:00:53.606 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47072.log 2024-09-10T21:00:53.606 INFO:teuthology.orchestra.run.smithi098.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37148.log.gz 2024-09-10T21:00:53.606 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45575.log 2024-09-10T21:00:53.607 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51296.log: gzip/var/log/ceph/ceph-client.admin.47072.log: -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-10T21:00:53.607 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45575.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47072.log.gz 2024-09-10T21:00:53.607 INFO:teuthology.orchestra.run.smithi098.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51296.log.gz 2024-09-10T21:00:53.607 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45575.log.gz 2024-09-10T21:00:53.607 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49126.log 2024-09-10T21:00:53.607 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48292.log 2024-09-10T21:00:53.607 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-mgr.y.log: gzip/var/log/ceph/ceph-client.admin.49126.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47835.log 2024-09-10T21:00:53.607 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48292.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49126.log.gz 2024-09-10T21:00:53.607 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T21:00:53.607 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48292.log.gz 2024-09-10T21:00:53.607 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47835.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47835.log.gz 2024-09-10T21:00:53.607 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46861.log 2024-09-10T21:00:53.607 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36420.log 2024-09-10T21:00:53.607 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46861.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46861.log.gz 2024-09-10T21:00:53.608 INFO:teuthology.orchestra.run.smithi098.stderr: 84.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T21:00:53.608 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42653.log 2024-09-10T21:00:53.608 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47607.log 2024-09-10T21:00:53.608 INFO:teuthology.orchestra.run.smithi098.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.36420.log.gz 2024-09-10T21:00:53.608 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51086.log 2024-09-10T21:00:53.608 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42653.log.gz 2024-09-10T21:00:53.608 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47607.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47607.log.gz 2024-09-10T21:00:53.608 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44829.log 2024-09-10T21:00:53.608 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50906.log 2024-09-10T21:00:53.608 INFO:teuthology.orchestra.run.smithi098.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-10T21:00:53.608 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44829.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51086.log.gz 2024-09-10T21:00:53.608 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43543.log 2024-09-10T21:00:53.608 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44829.log.gz 2024-09-10T21:00:53.609 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44231.log 2024-09-10T21:00:53.609 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50906.log: /var/log/ceph/ceph-client.admin.43543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48705.log 2024-09-10T21:00:53.609 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44231.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50906.log.gz 2024-09-10T21:00:53.609 INFO:teuthology.orchestra.run.smithi098.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-10T21:00:53.609 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49879.log 2024-09-10T21:00:53.609 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44231.log.gz 2024-09-10T21:00:53.609 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48705.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48705.log.gz 2024-09-10T21:00:53.609 INFO:teuthology.orchestra.run.smithi098.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.43543.log.gz 2024-09-10T21:00:53.609 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48957.log 2024-09-10T21:00:53.610 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36130.log 2024-09-10T21:00:53.610 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49879.log.gz 2024-09-10T21:00:53.610 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42953.log 2024-09-10T21:00:53.610 INFO:teuthology.orchestra.run.smithi098.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48957.log.gz 2024-09-10T21:00:53.610 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44189.log 2024-09-10T21:00:53.610 INFO:teuthology.orchestra.run.smithi098.stderr: 76.1% -- replaced with /var/log/ceph/ceph-client.admin.36130.log.gz 2024-09-10T21:00:53.610 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45365.log 2024-09-10T21:00:53.610 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5%/var/log/ceph/ceph-client.admin.44189.log: -- replaced with /var/log/ceph/ceph-client.admin.42953.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44497.log 2024-09-10T21:00:53.610 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-10T21:00:53.610 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44189.log.gz 2024-09-10T21:00:53.610 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50463.log 2024-09-10T21:00:53.610 INFO:teuthology.orchestra.run.smithi098.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45365.log.gz 2024-09-10T21:00:53.610 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46601.log 2024-09-10T21:00:53.611 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44497.log: /var/log/ceph/ceph-client.admin.50463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44487.log 2024-09-10T21:00:53.611 INFO:teuthology.orchestra.run.smithi098.stderr: 87.0% 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50463.log.gz 2024-09-10T21:00:53.611 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.2.46601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51212.log 2024-09-10T21:00:53.611 INFO:teuthology.orchestra.run.smithi098.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.46601.log.gz 2024-09-10T21:00:53.611 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44497.log.gz 2024-09-10T21:00:53.611 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45089.log 2024-09-10T21:00:53.611 INFO:teuthology.orchestra.run.smithi098.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44487.log.gz 2024-09-10T21:00:53.611 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44827.log 2024-09-10T21:00:53.611 INFO:teuthology.orchestra.run.smithi098.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51212.log.gz 2024-09-10T21:00:53.611 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44830.log 2024-09-10T21:00:53.611 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45089.log.gz 2024-09-10T21:00:53.611 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46987.log 2024-09-10T21:00:53.612 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44827.log.gz 2024-09-10T21:00:53.612 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43103.log 2024-09-10T21:00:53.612 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44830.log.gz 2024-09-10T21:00:53.612 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50379.log 2024-09-10T21:00:53.612 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46987.log.gz 2024-09-10T21:00:53.612 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48376.log 2024-09-10T21:00:53.612 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43103.log.gz 2024-09-10T21:00:53.612 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36607.log 2024-09-10T21:00:53.612 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50379.log.gz 2024-09-10T21:00:53.612 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48334.log 2024-09-10T21:00:53.612 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48376.log.gz 2024-09-10T21:00:53.612 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48873.log 2024-09-10T21:00:53.612 INFO:teuthology.orchestra.run.smithi098.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36607.log.gz 2024-09-10T21:00:53.612 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50117.log 2024-09-10T21:00:53.613 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48334.log.gz 2024-09-10T21:00:53.613 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51422.log 2024-09-10T21:00:53.613 INFO:teuthology.orchestra.run.smithi098.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48873.log.gz 2024-09-10T21:00:53.613 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50295.log 2024-09-10T21:00:53.613 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50117.log: /var/log/ceph/ceph-client.admin.51422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48663.log 2024-09-10T21:00:53.613 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50117.log.gz 2024-09-10T21:00:53.613 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51422.log.gz 2024-09-10T21:00:53.613 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48617.log 2024-09-10T21:00:53.613 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50295.log: /var/log/ceph/ceph-client.admin.48663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36166.log 2024-09-10T21:00:53.613 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48663.log.gz/var/log/ceph/ceph-client.admin.48617.log: 2024-09-10T21:00:53.613 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44105.log 2024-09-10T21:00:53.613 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50295.log.gz 2024-09-10T21:00:53.613 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48617.log.gz 2024-09-10T21:00:53.613 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36166.log: gzip -5 --verbose -- 76.6% -- replaced with /var/log/ceph/ceph-client.admin.36166.log.gz 2024-09-10T21:00:53.614 INFO:teuthology.orchestra.run.smithi098.stderr: /var/log/ceph/ceph-client.admin.50948.log 2024-09-10T21:00:53.614 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48418.log 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44105.log.gz 2024-09-10T21:00:53.614 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-10T21:00:53.614 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44832.log 2024-09-10T21:00:53.614 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50948.log.gz 2024-09-10T21:00:53.614 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44147.log 2024-09-10T21:00:53.614 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48418.log.gz 2024-09-10T21:00:53.614 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36076.log 2024-09-10T21:00:53.614 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44147.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44832.log.gz 2024-09-10T21:00:53.614 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46162.log 2024-09-10T21:00:53.614 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44147.log.gz 2024-09-10T21:00:53.614 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36076.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36076.log.gz 2024-09-10T21:00:53.614 INFO:teuthology.orchestra.run.smithi098.stderr: -- /var/log/ceph/ceph-client.admin.42153.log 2024-09-10T21:00:53.615 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.0.46162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46162.log.gz 2024-09-10T21:00:53.615 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-10T21:00:53.615 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-10T21:00:53.615 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51858.log 2024-09-10T21:00:53.615 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42153.log.gz 2024-09-10T21:00:53.615 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42330.log 2024-09-10T21:00:53.615 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45677.log 2024-09-10T21:00:53.615 INFO:teuthology.orchestra.run.smithi098.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51858.log.gz 2024-09-10T21:00:53.615 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45086.log 2024-09-10T21:00:53.615 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45677.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42330.log.gz 2024-09-10T21:00:53.615 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42869.log 2024-09-10T21:00:53.615 INFO:teuthology.orchestra.run.smithi098.stderr: 90.8%/var/log/ceph/ceph-client.admin.45086.log: -- replaced with /var/log/ceph/ceph-client.admin.45677.log.gz 2024-09-10T21:00:53.616 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45086.log.gz 2024-09-10T21:00:53.616 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44813.log 2024-09-10T21:00:53.616 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42869.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42869.log.gz 2024-09-10T21:00:53.616 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44813.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44813.log.gz 2024-09-10T21:00:53.616 INFO:teuthology.orchestra.run.smithi098.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-10T21:00:53.617 INFO:teuthology.orchestra.run.smithi190.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T21:00:53.665 INFO:teuthology.orchestra.run.smithi190.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T21:00:53.780 INFO:teuthology.orchestra.run.smithi098.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T21:00:54.140 INFO:teuthology.orchestra.run.smithi098.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T21:00:54.140 INFO:teuthology.orchestra.run.smithi098.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T21:00:54.170 INFO:teuthology.orchestra.run.smithi098.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T21:00:54.192 INFO:teuthology.orchestra.run.smithi190.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-10T21:00:54.212 INFO:teuthology.orchestra.run.smithi190.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-10T21:00:54.221 INFO:teuthology.orchestra.run.smithi190.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-10T21:00:54.239 INFO:teuthology.orchestra.run.smithi190.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-10T21:00:54.241 INFO:teuthology.orchestra.run.smithi190.stderr: 2024-09-10T21:00:54.241 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m0.723s 2024-09-10T21:00:54.241 INFO:teuthology.orchestra.run.smithi190.stderr:user 0m2.913s 2024-09-10T21:00:54.241 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m0.124s 2024-09-10T21:00:54.299 INFO:teuthology.orchestra.run.smithi098.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-10T21:00:54.300 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-10T21:00:54.300 INFO:teuthology.orchestra.run.smithi098.stderr:real 0m0.813s 2024-09-10T21:00:54.300 INFO:teuthology.orchestra.run.smithi098.stderr:user 0m2.911s 2024-09-10T21:00:54.300 INFO:teuthology.orchestra.run.smithi098.stderr:sys 0m0.248s 2024-09-10T21:00:54.301 INFO:tasks.ceph:Archiving logs... 2024-09-10T21:00:54.302 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899652/remote/smithi098/log 2024-09-10T21:00:54.303 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T21:00:54.921 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899652/remote/smithi190/log 2024-09-10T21:00:54.922 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T21:00:55.358 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T21:00:55.367 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T21:00:55.367 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T21:00:55.370 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T21:00:55.473 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T21:00:55.481 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T21:00:55.482 DEBUG:teuthology.orchestra.run.smithi098:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:00:55.484 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:00:55.498 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-09-10T21:00:55.502 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-09-10T21:00:55.503 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T21:00:55.504 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-09-10T21:00:55.504 INFO:teuthology.orchestra.run.smithi098.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 -391us[ -395us] +/- 45ms 2024-09-10T21:00:55.504 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 +2038us[+2038us] +/- 81ms 2024-09-10T21:00:55.504 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 -1204us[-1207us] +/- 75ms 2024-09-10T21:00:55.504 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:00:55.507 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T21:00:55.507 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-09-10T21:00:55.507 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 -374us[ -377us] +/- 45ms 2024-09-10T21:00:55.507 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 +2027us[+2027us] +/- 81ms 2024-09-10T21:00:55.507 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 -1217us[-1217us] +/- 75ms 2024-09-10T21:00:55.508 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:00:55.508 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T21:00:55.517 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T21:00:55.518 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T21:00:55.540 DEBUG:teuthology.orchestra.run.smithi098:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T21:00:55.616 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T21:00:55.680 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T21:00:55.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:00:55.709 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T21:00:55.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:00:55.737 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T21:00:55.746 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T21:00:55.764 INFO:teuthology.task.internal:Duration was 521.220125 seconds 2024-09-10T21:00:55.764 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T21:00:55.798 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T21:00:55.798 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T21:00:55.801 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T21:00:55.838 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T21:00:55.840 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T21:00:56.094 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T21:00:56.095 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi098.front.sepia.ceph.com 2024-09-10T21:00:56.095 DEBUG:teuthology.orchestra.run.smithi098:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T21:00:56.155 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2024-09-10T21:00:56.156 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T21:00:56.183 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T21:00:56.183 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T21:00:56.198 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T21:00:56.249 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T21:00:56.250 DEBUG:teuthology.orchestra.run.smithi098:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T21:00:56.265 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T21:00:56.391 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T21:00:56.401 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T21:00:56.402 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T21:00:56.427 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T21:00:56.452 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T21:00:56.462 DEBUG:teuthology.orchestra.run.smithi098:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:00:56.469 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:00:56.490 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = core 2024-09-10T21:00:56.516 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2024-09-10T21:00:56.530 DEBUG:teuthology.orchestra.run.smithi098:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:00:56.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:00:56.555 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:00:56.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:00:56.585 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T21:00:56.595 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T21:00:56.596 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899652/remote/smithi098 2024-09-10T21:00:56.596 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T21:00:56.631 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899652/remote/smithi190 2024-09-10T21:00:56.632 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T21:00:56.666 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T21:00:56.666 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T21:00:56.669 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T21:00:56.715 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T21:00:56.726 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T21:00:56.727 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T21:00:56.748 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T21:00:56.749 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T21:00:56.751 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T21:00:56.763 INFO:teuthology.orchestra.run.smithi098.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 21:00 /home/ubuntu/cephtest 2024-09-10T21:00:56.770 INFO:teuthology.orchestra.run.smithi190.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 21:00 /home/ubuntu/cephtest 2024-09-10T21:00:56.771 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T21:00:56.781 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T21:00:56.851 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/cap-flush} duration: 521.2201251983643 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-10T21:00:56.851 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T21:00:56.953 INFO:teuthology.run:pass